Marvin Johnson's stoppage-time goal at Inverness made it six wins in the last seven league matches for the Steelmen.
"Even if we end up in the top six we want to get as many points as we can," McGhee told BBC Scotland.
"That means the Celtic game is every bit as important as the rest.
Fourth-placed Motherwell will clinch a top-half place in the Premiership if Partick Thistle fail to beat Dundee United in midweek.
The Motherwell manager won the title as a player with both the Dons and Celtic and saw his side put a dent in the current Pittodrie club's hopes of success a fortnight ago.
That 2-1 defeat at Fir Park leaves Aberdeen playing catch-up but McGhee insists they will make it just as tough for the defending champions at the same venue next weekend.
He said: "Thistle are still coming up in the rails so we will see what happens to them in midweek and what we need from the Celtic game on Saturday.
"Nothing will lessen the importance of the Celtic game as we have to maintain the integrity of the competition as well.
"We have got the give them every bit as hard a game as we gave Aberdeen when we beat them before the international break.
"Hopefully we will be in the top six and we can make it a good end to the season but we are not taking anything for granted.
"Today was by far the poorest performance in that seven-game run as I feel we mugged Inverness today."
John Hughes is frustrated that his Inverness Caley Thistle players are still paying the price for naive defending late on in games.
The Highlanders could still sneak into the top six with wins against Hearts at home and away to Dundee United before the split.
Finishing in the bottom half of the table for the first time in four years looks much more likely though after losing to Johnson's sucker punch goal in the third minute of stoppage time.
He said: "We have lost too many goals late in games this season. Everyone wants to be a hero and win games but you need to be more intelligent at times.
"I'm honest enough to say I don't know what I'm going to get from the team now. We need to look at ourselves in the mirror and go out and win every game to the end of the season now.
"The boys work hard and they dominated the game but we keep beating ourselves as the goals we gave away were comical.
"If we keep doing that we are going to be right up against it. It's simple, we need to put the ball in the net more often and keep the back door shut."
Mark McGhee has promised Aberdeen that Motherwell will make every effort to beat Celtic at Fir Park next Saturday regardless of whether or not they have clinched a top-six place by then.