You might want to consider Gödel’s incompleteness theorems as a solution of sorts here.

1st theorem: No system of axioms whose theorems can be listed by an “effective procedure” (e.g. a computer program, but it could be any sort of algorithm) is capable of proving all truths about the relations of the natural numbers (arithmetic). For any such system, there will always be statements about the natural numbers that are true, but that are unprovable within the system. (Emphasis mine)

2nd theorem: An extension of the first, it shows that a system cannot demonstrate its own consistency.

I recognize I’m being very vague in my description of the second theorem. As I recall, these descriptions were scribbled in my notes while I was attempting to get through Douglas Hofstadter’s Gödel, Escher, Bach. Don’t ask me to explain how the second theorem manages to prove that a mathematical system can’t demonstrate its own consistency. I’m not a mathematician. None-the-less, to my mind at least, it’s self-evident that no system (mathematical or otherwise) can serve as evidence of its own truth (i.e. the Bible can’t be “true” simply because the Bible claims it is true).

That said, ultimately, all arguments are circular in so far as they rely upon reasons outside themselves for justification, which in turn rely upon reasons outside themselves, and so on. Sooner or later, if you follow these reasons back far enough, you end up where you began. Circular arguments are only a problem if the circle of reasons is too small.