Great question. This passage has often been used to justify certain types of universalism. There is nothing wrong with your logic as it pertains to this verse, but as you wisely pointed out, context is key.
The first question that we need to wrestle with is exactly how the sin of Adam correlates to my situation? On one hand, the introduction of sin into the world means that I am, as a human, a member of a race that is separated from God. I was born on the wrong side of the divide. On the other hand, and it seems to be the point of verse 12, that death (spiritual separation) entered our world through the sin of Adam which opened the door for the relationship between sin and death in our world. But, as v12 continues it was because ‘all sinned’ that this cycle continues. I guess in one sense, not only did Adam sin, but also serves as a paradigmatic figure for how humans behave. So sin entered through one man, but in a non-justified state I’m also separated from God because of my own sin. Scripture doesn’t allow me to blame shift for my own sin. So Adam simultaneously gives us an origin for the condition that we naturally find ourselves in, and at the same time, doesn’t give me an excuse, or a justification, for how I am.
Paul has been making a sustained argument (starting all the way back into chapter 3) about righteousness coming by faith and not categorically by association with Abraham/Israel. It isn’t enough to be part of a special group that identifies with Abraham, we must be saved in the same way Abraham was- by faith. In effect, Christ breaks the cycle of sin and death. There is a way to the other side. Death is no longer the predominate power. Grace now reigns (see v21).
This is a bit of a cheesy analogy, but it captures what Paul is perhaps pointing toward. Christ is taking a bus to the other side and has enough seats for us all, the question is do we want to be on that bus and can we get on it without His help? This is where the conversation is going in chapter 6 where Paul talks about us being unified with Christ by dying to sin, being buried and raised to new life through baptism. The conversation then goes on to discuss being slaves to righteousness since grace is reigning instead of death.
Your question strikes at the core of several significant theological divides, which we won’t delve into here particularly regarding baptism. The main point is that even those who do baptism services for infants still teach that there needs to be a cognitive embrace and life of faith, love, and loyalty expressed to God for Christ’s work to be effectual. There will be big divides here between people who have differing views on the relationship between the church and Israel.
One further note: In the Bible salvation is much bigger than just justification. The goal of our faith is not to be forgiven, it is to be restored in relationship with God (No one comes to the Father but by me…). Forgiveness is the prerequisite, but not the final destination. The justification that comes from Christ brings life (v18) and the fullness of that life is that we know the Father (Jn 17:3) which isn’t something that we lovingly enter into without knowledge or consent. Long answer short, to consistently read the context of the passage you are pointing to, you had no choice in Adam’s sin and you had no choice in Jesus’ victory over death, but because of the work of Christ you now have the choice of no longer living bound by sin, and, in Paul’s words to offer yourself (6:19) as a slave to righteousness which leads to freedom from sin and leads to holiness (6:22). We have the broader biblical truths that “The soul who sins is the soul who dies” Ezk 18:4. And ‘Everyone who calls on the name of the Lord will be saved” Acts 2:21. This loops us back around to pondering our first question about how you are connected to Adam’s sin, people may come out at different places on it, and thus have different understandings on this verse. I’m not sure how you define or think about ‘original sin,’ but my guess is that thinking through that will bring you some clarity on this verse. I don’t believe I’ve fully answered your question, but perhaps these thoughts will enable you to continue thinking in a helpful direction. You are quite right- we can get ourselves into all sorts of confusion by taking a verse out of context. Thanks for your clarifying question. Blessings~ Nathan