I think I have finally beat the mold. DH doesn't think it smells like mold anymore and my kids aren't saying that anything stinks so I guess I'm good to go. Thanks everyone for all your great suggestions!
I put as much of it as possible in my wash tub with bleach and scrubed and soaked for awhile and did that for most of the day, turning as needed. Then I set it out to dry and I put some baking soda in there as well. It's good to go now.