My son had the solid 'signs' at 4 months but we waited until he was 6 months as he was still sleeping well and gaining lots of weight.
Can you start giving food straight away? - sure if they have the right signs and the right reflexes for eating. There is a lot of info out there and I am sure your clinic nurse has lots of helpful tips.
First food? - yep it usually is the rice cereal. This is because it is fairly bland and has the right nutrients needed for baby and the right ingredients suited to their immature palates and kidneys etc which is only used to fluid and not solid food.
Can you stop breastfeeding? - that is up to you. Whatever you are comfortable with or what you think your baby needs. The literature out there states the longer you breastfeed the better for the baby. The main source of food for baby up to 10 -12 months is breastmilk or formula so even if you stop breastfeeding, you still need to give him formula as well as solids. Check out the heinz website as it is quite helpful for this sort of stuff.