Summer tyres are designed for summer use and the rubber will become hard if used during minus degrees. Winter tyres have been designed for winter use, so the rubber will remain soft and agile despite minus degrees.
This means that you need to shift to winter tyres when the temperature drops. However, if you have winter approved all weather tyres on your car, you can keep driving. Do remember that with all-weather tyres you should rotate the tyres every 2000km to ensure even wear between the tyres.