The writer is talking about confidence and assurance in the unknowns. One of the things I have learned in my faith journey--particularly in those darkest days after my wife and son died--is that God is not afraid of my questions. He can handle them.
The difference between the questioning believer and the cynic who refuses to believe is where you place your focus. I can choose to focus on questions that seemingly have no answers. Many of those are simply questions that will not be answered this side of heaven. Others are answered only by the supreme wisdom of God. But such a focus only leads to cynicism that destroys.
On the other hand, I can focus on what I know to be real: that the evidence is overwhelming that the Bible is real (from archaeology and historical documents), that a resurrected Jesus appeared to over 500 witnesses, that science continues to prove the necessity of an Intelligent Designer for creation. The questions are not what keeps me from believing in God. On the contrary, they are the things that remind me that He is God...I am not.
Don't be afraid to ask your questions. And, don't be afraid when all the answers are not readily available. "Seeing is believing" may be good for some things but, when it comes to faith, Hebrews reminds us that the unseen things are constant reminders of a God who is bigger than all of life.