Storey, on loan from Swindon Town, put the hosts ahead inside the opening 15 minutes, drilling a low shot into the bottom corner.
Kgosi Ntlhe's chipped finish was ruled out for offside after the break as Peterborough pushed for an equaliser.
Striker Craig Westcarr also had a goal disallowed, but Pompey held on to beat their third-tier opponents.
The result means Portsmouth progress to the second round of the competition for the first time since 2010.
Miles Storey's early goal helped League Two Portsmouth shock Peterborough in the first round of the League Cup.