Do Americans feel it difficult to understand British/Aussie accents?

I'm Chinese. You know, in China there are numerous dialects, which sometimes make one hard to understand another one's accent. As to English, I can understand American accent, but feel hard to understand Aussie accent and British accent. And regarding Indian dialect, I can't comprehend even one word...