The manifold is also known as the header, and shouldn't be confused with the intake manifold. It's an arrangement of metal pipes that bolt to the exhaust side of the engine, and there is commonly one pipe per cylinder.
Exhaust manifolds are frequently made of either cast iron or steel. Between the manifold and the engine is a manifold gasket which maintains a gas-tight seal between the components.
There are several different manifold designs but the purpose is always the same: to channel the spent exhaust gases into a pipe (or pipes) that run down the length of the car.
Most cars have a downpipe - it connects the manifold to the catalytic converter (also known as the cat). On some cars the cat is integral to the downpipe, and on others there are 'pre-cats', which as the name suggests are catalytic converters that aid the function of the main cat.
The downpipe commonly (but not always) features a 'flexi' section (this is most common on a front-wheel-drive transverse engine set-up). This articulated section of the pipe allows the engine to move without risking the vibration damaging the manifold.
We all know that exhaust gases are a pollutant and contain some pretty unpleasant substances. The catalytic converter's task is to convert toxic gases into less toxic gases.
The cat features a honeycomb-like structure which increases its surface area. The catalysts within the converter break up nitrogen oxides into nitrogen and oxygen (which are harmless once separated).
They then convert carbon monoxide into carbon dioxide - and finally they convert unburnt hydrocarbons into carbon dioxide and water. The result is emissions that, while not totally harmless, are less harmful than the gases entering the cat.
As you drive the exhaust moves (which is why engine mounts allow for movement). If the exhaust was bolted directly to the underside of the car the movement of the engine could easily crack, and damage the exhaust.
To prevent this exhaust systems are suspended via a series of rubber (or polyurethane) hangers. These can often fail before the exhaust, and will cause your car to fail its annual Mott test, so it's worth checking yours periodically.