Leicester City 1 Barnsley 2
The former Sunderland striker put the Tykes ahead after twelve minutes after some fine build-up play from Ricardo Vaz Te, before Lloyd Dyer responded with an equaliser three minutes later.
Gray restored Barnsley's lead ten minutes before half-time with a close-range header after the Foxes were caught on the counter-attack.
Leicester are now winless at home since November last year, a run of league form which has seen them slip to fifteenth in the Championship.
The home side almost took the lead early on when Lloyd Dyer was brought down on the edge of the area by Stephen Foster, but Paul Gallagher's low free-kick was turned away by Barnsley goalkeeper Luke Steele.
That save turned out to be a crucial point in the match as five minutes later, Danny Drinkwater and Ricardo Vaz Te worked a counter-attack to great effect before Vaz Te stepped past Matt Mills and found the unmarked Andy Gray in the penalty area, who had the simplest of tasks in slotting the ball past Kasper Schmeichel.
It was a well-worked counter attack and emphasised Leicester's failure to close the opposition down.
The Foxes were back in the match three minutes later, however, when Matt Mills' superb long pass picked out Lloyd Dyer, who controlled the ball well between two Barnsley defenders and fired across Luke Steele from twelve yards.
It was a well-executed finish by the winger, but the pinpoint long ball from Mills was outstanding and demonstrated the captain's ability to distribute play from deep.
The half began to fizzle out after the equaliser, but Barnsley restored their lead ten minutes before half-time when Vaz Te and Drinkwater again combined on the counter-attack on the left side, with the Manchester United loanee's cross picking out Gray again in the area, who headed past Schmeichel.
It was a simple yet devastating passage of play to undo Leicester once again, questioning their defensive capability.
After the break, Barnsley should have doubled their advantage when Matt Mills' calamitous back pass was tracked down by Ricardo Vaz Te but somehow cleared by Kasper Schmeichel with the forward breathing down his neck.
At the other end, Neil Danns fired over for the Foxes before Lloyd Dyer threatened the Yorkshire side with some neat footwork, but the attack was eventually thwarted by a resolute Barnsley backline.
With just over twenty minutes remaining, Andy King had a great chance to test Luke Steele when David Nugent rose to head the ball down to the Welsh midfielder inside the penalty area, but his wild shot was high and wide, much to the frustration of the fans behind the goal.
Barnsley continued to dominate the match as Jay McEveley's mazy run from defence tore through the Foxes' midfield before the full-back's clinical pass found substitute Matt Done one-on-one with Schmeichel, but his shot was beaten away by the Danish shot stopper.
Substitute Jermaine Beckford sparked some life into Leicester's attack, but his best chance was well-blocked by McEveley at point-blank range.
Vaz Te almost added insult to injury in the closing stages of the match when the Portuguese forward raced clear of two Leicester defenders, but his cheeky finish was saved by the feet of Schmeichel.
The final whistle lead out a chorus of boos from all areas of the ground, apart from the travelling Barnsley supporters who climb above Leicester in the Championship following their win.
Nigel Pearson's side have now won just three out of ten games since his return to the club, taking thirteen points from a possible 30.