I ride Carl R. Bieber, Inc. twice a day, three days a week. Not all of the busses are in perfect condition, and sometimes a seat is broken. This morning, the seat in which I sat had a broken recliner handle. Even if I didn't touch it, any pressure on my seat back would cause it to recline. For a while, I tried to remain upright. When that got difficult, as I worked, I moved to the aisle seat. There was a guy behind me, and there was no reason to make him try to cope with my arbitrary back and forth movements.
I don't think that it's necessarily rude to lean your seat back when there's someone behind you. It's just the sort of thing about which you should ask first. "Excuse me," you can ask, "would you mind if I leaned my seat back a little?"
It's not hard. You might even make the person behind you feel good about being asked.
Despite that, I am now folded up into a bundle of acute angles after the sudden encroachment of the seat in front of me. At least I'm in the extra leg room row.