Simply put, the problem I have here is that a great amount of the criticism I am seeing directed at these cartoons is coming from people of colour. In my case, simply because of the directions my whims have taken me, the criticisers are also by and large American. And I have a huge issue with the white people I've seen defending Charlie by telling black Americans they can't recognise racism when they see it because they're not French. It's just too easy and glib and frankly patronising an answer, and not one black Americans are used to hearing in far too many similar forms.
I'm not saying there's no chance nuance is being lost, of course. And I'm aware that as a white Englishman I can't speak with no greater authority on racial theory as espoused by non-white people than I can on the satirical history of our Gallic neighbours. I can say that the position that using the imagery of racism to mock racists is still hugely problematic is a common-held and entirely considered one, though, and that in everything I've seen - including some of the links Tomsk offers- that stance is being ignored or straw-manned so that black people can be lectured for "not getting it". Pointing that out isn't "Anglosplaining", it's simply noting that an argument has been made that I'm not seeing anyone refute. Various deconstructions of the French humour mechanism that generated these cartoons are on offer, but as to why drawing a black woman as a monkey is cool if you're doing it to mock racists, I'm still none the wiser. 
There's also the issue of Islamophobia to consider (the degree to which one cannot help but be racist if one demonstrates Islamophobic tendencies is an exercise I leave to the reader). The suggestion Charlie is Islamophobic is far from limited to non-French people; even current/former employees of the magazine have taken exception to its treatment of the religion after 9-11. I read an article this weekend that I'm desperately trying to locate from a former Charlie
Lastly, it should be remembered that one can hold explicitly anti-racist stances and still produce racist material. Indeed, this is essentially inevitable. Even worse, it can be more common for those of us on the left to do so than others, because we make mistakes whilst trying to engage with the problem whilst others ignore it entirely (I saw a depressing synecdoche for this on Twitter a couple of months ago when a black woman explained most of the racist tweets she got were from progressives, because they try and show solidarity in offensively incompetent ways, whilst at least the conservatives just completely bypass her). Hell, I've based this entire post on the theory that I have a half-decent working understanding of why so many black Americans are furious over these cartoons, which itself might be presumptuous of me to the point of racism. 
In short, I can completely respect an argument that says Charlie Hebdo is dedicated to the fight against racism. I can respect an argument that says given that fact, we shouldn't rip apart its missteps in this regard, and recognise that taken holistically CH is a force for good (though as a white guy I don't get to decide how much anti-racist work counteracts even one racist cartoon). But defending these specific cartoons is a much harder job than I've seen anyone inclined to do so willing to admit to, and that job doesn't even seem to be being seriously attempted because their so-called defenders are more interested in building a hierarchy of authorities on recognising racism. Which, as I've noted, just so happens to have a bunch of predominantly white people at the very top. Plus ça change, plus c'est la même chose.
 See also the Suey Park/Stephen Colbert controversy last March, in which Colbert's twitter feed was jumped on after he mocked the Washington Redskins franchise's refusal to change its name by announcing he would "show the Asian-American community I care by starting the 'Ching-Chong Ding-Dong Foundation for Sensitivity to Orientals or Whatever'". My point here is not that Suey Park was clearly in the right, but that the argument that satirising bigots by appropriating their rhetoric (linguistic or visual) is something those on the shitty end of that bigotry can have a problem with. That's what needs addressing here, not whether this kind of appropriation is more common in France than elsewhere.
 It strikes me that at least some of the back-and-forth here stems from the conflation of "being racist" and "being a racist". I'm only interested in the first formulation, and a lot of the defences mounted on behalf of the magazine is that their staff clearly aren't racists. To be clear, I am criticising the cartoons, not the artists. I am insisting that everyone on the left - myself included - is capable of being racist, not that we're all racists. This is hardly an original point, but since I'm on the subject I shall bring it up once more: one consequence of our culture concluding racists are terrible people is that it's led most of us to assume we can't be racist, because we're clearly not a racist. The truth of course is vastly more complicated.