The Swans are bottom of the Premier League having won only one of Bradley's first seven games in charge.
"I don't spend any time thinking about it [my future]," the American said.
"If some people want to criticise - there were some who had me out the door from the first day because of my accent."
Bradley, the first American to manage in the Premier League, added: "They can hit the road because that doesn't have anything to do with anything."
He says he is already planning for the January transfer window and hopes to improve his squad "from back to front".
"When you're a manager, it goes with the territory," the 58-year-old said. "If you worry about that and spend time covering yourself in those ways, players will spit you out so fast, they'll see through it.
"If I am telling players 'you have to look adversity in the eye and you have to have some courage', I have to have it too."
Appointed in October after Francesco Guidolin was sacked, Bradley has endured a chastening start to his tenure, with Swansea now two points adrift at the foot of the Premier League table and three points from safety.
Chairman Huw Jenkins says he "accepts responsibility" for the Swans' slump, which has been compounded by problems off the field following the club's American takeover in the summer.
Jenkins has also said American owners Jason Levien and Steve Kaplan will make funds available during the January transfer window, and Bradley has been discussing potential signings with all three.
"We need to improve from back to front," he said.
"I have good discussions with Huw on all aspects throughout week. I also have a weekly call with Jason and Steve.
"If we come up with the right player and that player is eager to get to Swansea and feels it is a challenge they are up for, I have every bit of confidence Jason and Steve are ready to jump in and make moves that are going to help us."
Swansea face a crucial fixture in their bid to avoid relegation on Saturday when they host Sunderland, who are second from bottom of the Premier League.
Defender Federico Fernandez and midfielder Ki Sung-yueng have been ruled out because of injury.
Fernandez's absence is untimely given Swansea's defensive failures this season, conceding 31 goals so far - more than any other team.
Nineteen of those goals have been conceded in Bradley's seven games in charge, and the former United States and Egypt manager has warned his players they must improve.
"We have to defend better in all areas," he added. "We're not going to survive if we continue to concede that number of goals."
Swansea City's manager Bob Bradley has claimed that some critics want him out because of his accent.