In a first century account by Tacitus of the great fire of Rome during the reign of Emperor Nero, we find the following statement.
Consequently, to get rid of the report, Nero fastened the guilt and inflicted the most exquisite tortures on a class hated for their abominations, called Christians by the populace. Christus, from whom the name had its origin, suffered the extreme penalty during the reign of Tiberius at the hands of one of the procurators, Pontius Pilatus, and a most mischievous superstition, thus checked for the moment, again broke out not only in Judea, the first source of the evil, but even in Rome, where all things hideous and shameful from every part of the world find their center and become popular.
Two points to make note of:
1. Tacitus is a recognized accurate historian from the ancient world.
2. His reference to a "mischievous superstition" is generally accepted to be a reference to the resurrection of Christ.
In a letter to Emperor Trajan around A.D. 112 Pliny the younger discusses a Christian worship service.
They were in the habit of meeting on a certain fixed day before it was light, when they sang in alternate verses a hymn to Christ, as to a god, and bound themselves by a solemn oath, not to do any wicked deeds, but never to commit any fraud, theft or adultery, never to falsify their word, not deny a trust when they should be called upon to deliver it up; after which it was their custom to separate, and then reassemble to partake of food-but food of an ordinary and innocent kind.
We now move on to a controversial quote from the historian Flavius Josephus
Now there was about this time Jesus, a wise man, if it be lawful to call him a man, for he was a doer of wonderful works, a teacher of such men as receive the truth with pleasure. He drew over to him both many of the Jews, and many of the Gentiles. He was the Christ, and when Pilate, at the suggestion of the principal men among us, had condemned him to the cross, those that loved him at the first did not forsake him; for he appeared to them alive again the third day; as the divine prophets had foretold these and ten thousand other wonderful things concerning him. And the tribe of Christians so named from him are not extinct to this day.
This passage is referred to among scholars as the Testimonium. The italisized portions are suspected to have been added to the original writing. The debate among scholars comes down to three choices. Accept the passage entirely, reject the passage entirely or accept the passage partially. The third choice seems the most logical because there is another version of the testimonium from an Arabic manuscript that reads as follows.
At this time there was a wise man who was called Jesus. And his conduct was good, and [he] was known to be virtuous. And many people from among the Jews and the other nations became his disciples. Pilate condemned him to be crucified and to die. And those who had become his disciples did not abandon his discipleship. They reported that he had appeared to them three days after his crucifixion and that he was alive; accordingly, he was perhaps the Messiah concerning whom the prophets have recounted wonders.
There are numerous other references that I could quote but I think the point is made. There are legitimate, well known, thoroughly researched first and second century extra-biblical documents that support many of the gospel accounts and many of the practices of the early Christian church.
It should also be pointed out that these are writings from Authors unfriendly to Christianity. Their stories support, in a sometimes hostile tone, what Christians did and what they believed. Not once is evidence shown to counter the claims of the early Christian church.