I do not have a problem with Christians who sincerely disagree with young earth creationism (YEC) based on their understanding of Scripture. I do have a problem with Christians who reject young-earth creationism because they are afraid of what people will think of them. That is putting the fear of man over the fear of God, and it is idolatry. Christians engaged in this sin must repent.
Matt Walsh recently came out against young-earth creationism and took some significant heat for it. After following Walsh for years, I am 100% convinced he is in the former camp, not the latter. He has never shown fear of man in taking positions. Christians who reject YEC because they are afraid people will think they are uneducated rednecks are sinning against God. The key here is to act out of faith, not out of fear.
This means the argument that a literal six-day creation is a stumbling block is not a Biblical argument. Either the literal six-day creation is true or it is not, and it simply does not matter if it is a “barrier” to people coming to faith. (That, of course, does not even get into the doctrine of election.) There are other doctrines that, examined from man’s perspective, are far more “insane” than YEC. The most “insane” one in my opinion is the foundation of our faith, that God the Creator came to earth and died for man’s sins against Him. We cannot worry about what the world thinks of us. Our only concern is obeying our Lord and believing His Word.
As far as the Genesis creation story not being literally true but being some sort of poetic language, I can see that argument being used up to the creation of Man. The problem after that is that if you throw out the fact that Adam an Eve were real people you have to deny critical aspects of theology presented by the Apostle Paul in the New Testament. The Apostle Paul teaches us (in writings inspired by the Holy Spirit, so it is God’s Word) that by one man (Adam) sin entered the world and by one Man (Jesus Christ) the world was offered redemption from sin. Even if you want to argue that the rest of the universe was created over billions of years and designed by God, the New Testament requires that Adam and Eve were real people.