Atak Lual scored the only goal of the game in the 52nd minute in Juba the to give his side an historic win.
The result lifts South Sudan up to second in the table in Group C, with Benin and Mail playing on Sunday.
South Sudan played their first official friendly against Uganda in July 2012.
It was only in 2011 that organised football was first played in South Sudan, after the country gained its independence from Sudan.
On another historic day, Liberia pulled off a shock 1-0 victory over Tunisia in Monrovia, as the country hosted its first international football match since the lifting of an ebola ban.
Francis Forkey Doe scored the goal 11 minutes from time in difficult, wet conditions, to give his country their first points in Group A.  Togo lead the Group A standings after their victory in Djibouti on Friday.
South Africa have a big battle on their hands if they are to qualify for Gabon 2017, after a 3-1 defeat away to Mauritania.
Having been held to a draw by The Gambia in their first group M game, Bafana Bafana were under pressure to come away with a win.
They went behind after just six minutes through an Aly Abeid goal, and were then reduced to ten men when Siyabonga Zulu was sent off.
Thamsanqa Gabuza hit an equaliser for South Africa, but the hosts came back through Boubacar Beyguili and Moulaye Ahmed to seal a famous win.
Uganda edged to victory away to Comoros in Group D thanks to a single goal which came in the 25th minute.
Tonny Mawejje struck the decisive blow to lift the Cranes top of the table on a maximum six points from two games.
Uganda almost missed the tie because of financial difficulties but Ugandan President Yoweri Kaguta Museveni provided $210,000 to cover the team's travel costs.
In the group's other tie, Botswana beat Burkina Faso 1-0.
Joel Mogorilosi scored five minutes after the interval and his side held on to the advantage over the 2013 beaten finalists to secure their first points.
Congo Brazzaville went top of Group E after a 4-2 victory away to Guinea Bissau.
Ferebory Dore scored all four Congo goals.
In Group J Seychelles took their first point after a 1-1 draw with visitors Ethiopia.
Nelson Laurence slotted home a 24th minute penalty to give the hosts a surprise lead which lasted half an hour. A strike by Tesfaye Seyoum restored parity and put Ethiopia top of the table from Algeria who travel to Lesotho on Sunday.
Burundi hoisted themselves from the bottom of Group K with a 2-0 win over visiting Niger. Fiston Abdul Razak grabbled the opener on 24 minutes and Papy Faty  sealed the win with seven minutes of the tie remaining.
The result allowed Senegal top pull three points clear at the summit as they earned a 2-0 away win at Namibia. English Premier League stars Cheikhou Kouyate, of West Ham, and Southampton's Sadio Mane got the goals.
Three-time African champions Nigeria were fortunate to earn a point away to Tanzania in Group G, just about keeping their opponents out in a 0-0 draw.
Stand-in goalkeeper Carl Ikeme of English side Wolves, in for the bereaved Victor Enyeama, was the outstanding player of the match.
Wakaso Mubarak scored an 85th-minute free-kick to give Ghana a 1-0 victory over Rwanda in Group H. The Black Stars were without striker Asamoah Gyan because of a knee injury but managed to ensure they have maximum points from their first two games.
Morocco maintained their winning start to Group F, with a 3-0 victory away to Sao Tome e Principe.
Goals from Nordin Amrabat, Youssef El Arabi and Nabil Dirar sealed the victory for the North Africans.
South Sudan have achieved their first competitive victory since joining international football in 2012, beating Equatorial Guinea in their 2017 Africa Cup of  Nations qualifier.