Advice on planting herbs late summer (Canberra)?

Oregano and thyme should be OK as long as they get lots of sun. These plants are perennials, meaning they grow continually, at least for a few years. Capsicum is grown as an annual plant and usually dies after it has flowered and fruited. Apart from warmth, it also needs a certain amount of sunlight per day to flower properly (meaning it needs the longer days of spring and summer). Best to wait until late spring to plant one (say, November onwards). You could try growing some parsley (it's bulletproof!), coriander and lettuces.