An Autonomous Ground Vehicle (AGV) should be capable of self-navigating through various terrains based on priori data as well as self-configuring and optimizing its motion on the basis of sensed data. Research is in progress to improve terrain perception for planning, execution, and control of desired motion of an AGV. During the perception phase multiple classification techniques are used depending on underlying sensing technology. Obstacle detection in case of a compositetyped terrain is a challenging task because in order to apply classification the image has to be known as a single type. Image segmentation and then classifying each image-segment separately can help AGV proceed in the same direction (by selecting another path) even if it detects an obstacle in the image. This paper proposes a fuzzy classification scheme for terrain identification and obstacle detection to improve self-organization according to terrain type. In order to take an accurate decision, classified objects coming from the perception phase of different sensors need to be fused into a single accurate representation for both the environment and the obstacle. Moreover, we provide means for intelligent decision making in the selection of sensors, fusion of sensor data, assessment of obstacle state and direction. Finally, the evaluation of a recommended decision has been performed for the vehicle speed and direction.