Critics of the traditional Christian view of Luke's census often make claims that, if true, should have left traces in the church fathers and other ancient sources that aren't often discussed in this context. If Luke didn't intend to give a historical account when he wrote about the census, then some of Luke's contemporaries, probably many of them, should have known about it. If some of the earliest Christians, represented by Luke's gospel, believed that Jesus was born around ten years later than Matthew's gospel claims, we would expect some ancient Christians and non-Christians to have noticed that inconsistency. If some Christians were claiming that Jesus was born under Herod the Great, while others were claiming that He was born under Herod Archelaus, we would expect that difference to be noticed. Since people living in the earliest generations of Christianity frequently experienced censuses, we would expect them to notice if Luke was describing census procedures that would be implausible. Etc.
Ancient Christianity had a wide variety of enemies. As Irenaeus reminds us, a heretic who accepted one of the four gospels wouldn't necessarily accept another (Against Heresies, 3:11:7). Just as a pagan or a non-Christian Jew would have an interest in criticizing inconsistencies between the infancy narratives of Matthew and Luke, so would a Marcionite or Ebionite.
And we shouldn't assume that the earliest Christians wouldn't have noticed and discussed such issues themselves. When Augustine discusses alleged errors or inconsistencies in the infancy narratives in his Harmony Of The Gospels, it's sometimes unclear whether he's addressing the arguments of non-Christians or questions that he or other Christians had raised. Ancient Christians, like modern Christians, were capable of critiquing their own beliefs, and we shouldn't assume that they would have been too dishonest to address questions that arose surrounding the census. Documents like Justin Martyr's Dialogue With Trypho and Origen's Against Celsus were written because Christians had an interest in addressing criticisms of their belief system.
As early as the second century, we see Christians producing harmonies of the gospels, because they were thinking about such issues. Tatian's gospel harmony is the one most often discussed, but some scholars believe that Justin Martyr worked with a gospel harmony of his own even earlier. Craig Allert argues that there were a few other gospel harmonies circulating in the earliest centuries as well, such as one attributed to Theophilus of Antioch and another attributed to Ammonius (A High View Of Scripture? [Grand Rapids, Michigan: Baker Academic, 2007], pp. 114-116). Euebius of Caesarea hyperbolically refers to how "every believer" offered an explanation for the differences between the genealogies of Jesus in Matthew and Luke (Church History, 1:7:1). The early Christians were willing to dispute the canonicity of books like 2 Peter and Revelation, so it doesn't seem that they accepted the canonical books, like the gospels, uncritically. If Matthew and Luke were giving radically contradictory accounts of the events surrounding Jesus' infancy, such as the census, the earliest Christians, including contemporaries of the apostles and people who lived just after their time, don't seem to have been aware of it. The historical record suggests that there wasn't a separation between an original Matthean community and an original Lukan community. Rather, the two gospels were accepted together as harmonious from as far back as we can trace the issue. See, for example, Bruce Metzger's discussion of the widespread early acceptance of both gospels in The Canon Of The New Testament (New York: Oxford University Press, 1997). Given Luke's apostolic ties and his positive portrayal of the apostles and other early church leaders in Luke and Acts, it makes sense that he and Matthew would hold similar views.
As I've mentioned in previous articles (here and here), the gospels of Matthew and Luke were widely accepted and interpreted as giving harmonious historical accounts early on. A small minority of sources, such as Marcion and his followers, would reject one or both of the two gospels on other grounds (not because of inconsistencies related to the census), but the large majority of professing Christians accepted both.
The earliest interpreters of Luke's account refer to the census as a historical event: Justin Martyr (Dialogue With Trypho, 78), Clement of Alexandria (The Stromata, 1:21), etc. In the process of putting together this series, I contacted some New Testament and patristic scholars to ask some questions about how the census was viewed during the earliest centuries of church history. None of the scholars I corresponded with were aware of any ancient Christian or non-Christian source who questioned or denied the historicity of Luke's census. Julian the Apostate, a Roman emperor who wrote against Christianity in the fourth century, argued:
"Even Jesus, who was proclaimed among you, was one of Caesar's subjects. And if you do not believe me I will prove it a little later, or rather let me simply assert it now. However, you admit that with his father and mother he registered his name in the governorship of Cyrenius."
Apparently, Julian didn't argue against the census, he was at least willing to accept its historicity for the sake of argument, and he expected the average Christian to accept its historicity. Justin Martyr and Origen mention the census as a historical event in their responses to Trypho and Celsus, but neither saw any need to interact with arguments against the census, even though they address a wide variety of other arguments against Christianity.