If you spend the extra time needed (re-listening, working through transcripts, learning vocabulary, listening again) to understand what’s going on, then I wouldn’t say it’s too hard. Of course, it’s very hard to draw the line, but I would say that if you’re unable to understand the gist after listening several times, it’s too hard. Likewise, if you don’t plan to work with it at all, you need to understand a little bit of what’s going on, otherwise you’re not learning much and you will quickly tire (nothing is more exhausting than really trying to understand something and constantly failing). Check my article about the grand listening cycle.