Caffeine also has another important effect not discussed here:

Acetylcholine is a neurotransmitter. When excreted into your brain, it causes nerve impulses. Acetylcholine esterase is an enzyme that normally degrades acetylcholine and prevents the brain from maintaining an excited state for unnecessarily long times. Caffeine messes this all up by inhibiting the action of acetylcholine esterase. Thus, your neurons just keep on firing.