That drum is pretty tough. You won't break it if you have to get a little rough. With the nut and the drive shaft flange removed, the drum will slide off. Much like the other drums, the shoes may be holding you on. If you haven't, disconnect the hand brake cable. This may free you up a bit.
Tim