Do I leave the woman I had children with or stay?
OK, so I'm in a position where every move or option I have to take seems like the wrong one. I currently have two kids with a woman that I'm unequally yoked with. I can't see myself marrying this woman, so I'm thinking about moving out, but then I don't want to be away from my kids. What should I do?
You are in this position because you decided to sin. "Marriage is honorable among all, and the bed undefiled; but fornicators and adulterers God will judge" (Hebrews 13:4). You are not yoked with this woman, you are only living with her and bedding her -- that is the problem.
You cannot continue living in sin, so the two choices are either to marry the mother of your children so that you are no longer sinning, or move out. If you move out, it does not eliminate your responsibility to the children you sired. You still need to see that they are taken care of and to be involved in their lives.
This is a good illustration of why God said sex is not to take place outside of marriage. By being indecisive about what to do, you are indirectly excusing yourself for perpetuating your sins.