Oblivion's DLC installers are incompatible with 64-bit.
Of course you can just set the Universal Extractor on them and get the .esp/.bsa files, without going through the whole antipiracy security routine. Awesome copy protection work, guys!
You may run into some such pesky issues from time to time, even with 32-bit applications.
That said, you can always have dual-booting to go back to 32-bit quickly when you actually need it.
One last thing to know is that performance-wise, 64-bit is a bit slower. More overhead. Of course it's quickly mitigated for any resource-hungry application where the gain in data access time from having less need to swap stuff between RAM and disk more than compensate for slightly slower CPU instructions. So if you do stuff that require a lot of CPU use but not that much memory, you can as well stay on 32-bit.