Moeen made his Test debut in 2014 and went on to score 2,914 runs in 64 Tests at an average of 28.29, picking up 195 wickets.
The 35-year-old announced his retirement from the longest format of the game in September last year but in June said McCullum had sounded him out about a potential red-ball return.
England’s Test side next play a three-match series against Pakistan in Rawalpindi, Multan and Karachi in December but Moeen said he would not be with them.
“I’ve had an honest chat with Baz (McCullum) and I can’t see myself being stuck in hotels for another month and playing to the best of my ability,” Moeen wrote in his column for the Daily Mail on Monday.
“Baz phoned me, we spoke at length and I said, ‘Sorry, I’m done’. He understands, he knows the feeling. Test cricket is hard work. I’m 35 and something’s got to give.
“I want to enjoy my cricket and it wouldn’t be fair to reverse my decision and then struggle to give it my all. It’s time to close the door on that side of my career. To play 64 Tests for England has been a privilege and a dream fulfilled.”
Moeen captained England’s Twenty20 side in their 4-3 series win over Pakistan in the absence of the injured Jos Buttler, and will be a key player for the team in this month’s T20 World Cup in Australia.