Kargbo senior has always denied 2014 allegations of match-fixing made by the Sierra Leone Football Association.
The 17-year-old's father is a former Leone Stars captain and now plays for non-league English side Dulwich Hamlet.
"I really don't know if I'd play [for Sierra Leone] as what they've done to my dad is not nice," he told BBC Sport.
"You never know we'll see what God wants."
Kargbo Jnr could also play for Belgium after he moved to Europe at the age of eight to join his father who was playing professionally and secured citizenship.
Father and son were talking after a pre-season friendly on Tuesday between their two teams although the pair did not take to the pitch at the same time.
Kargbo senior says he will not pressure his son about any international commitments he has to make in the future.
"What they've done to me is bad but I won't decide my son's future. He can decide his future for himself," Kargbo senior insisted.
"My wish for him as a father is to be in a place where he'll be happy. If playing for Belgium will make him a globally successful player then why not."
"Everyone wants to see his son successful and become a winner."
"But you never know what'll come out of Sierra Leone. Maybe things will change for good but I don't see it coming soon."
Kargbo senior enjoyed the bragging rights on Tuesday as his Dulwich Hamlet side won the match 2-1.
"See him playing against my team is also something special for me. This is something I'll never forget in my life," he added
"Having a son like this developing so fast makes me a proud dad."
Kargbo Junior paid tribute to his father as well
"I've learnt a lot from my dad because he's like a role model to me," he added.
"I have always wanted to learn from him as he's been there and he knows what it takes to be a professional.
"What I admire about him is that he behaves like a professional on and off the pitch and I'm really proud of him."
Kargbo junior joined Crystal Palace last season from Reading and has been playing for their under-18 team.
"My target is to play Premier League football this coming season for my team even if it's for few minutes, that would be really good for me," he said.
"I want to score lot of goals whether that's for the Crystal Palace under-18, under-23 or first team. I know I'm capable of doing it for the first team."
Crystal Palace's Ibrahim Kargbo Junior admits match-fixing allegations against his father are making him doubt whether he would ever play for Sierra Leone.