It is very wrong to think that God would reject the prayers of someone in the state of mortal sin. The image of God as “Good shepherd” or as a loving Father is diminished when one is overly scared that God won’t answer their prayers because they are in sin. God is a loving father who is always willing to enter into dialogue with us so that our relationship with him might begin to heal. He never ever turns his back on his children. Everything depends on the disposition of the person who prays, not only about sin but how humble and contrite they are.
Here’s what the Church says on this matter:
When it arises from a love by which God is loved above all else, contrition is called “perfect” (contrition of charity). Such contrition remits venial sins; it also obtains forgiveness of mortal sins if it includes the firm resolution to have recourse to sacramental confession as soon as possible.
The contrition called “imperfect” (or “attrition”) is also a gift of God, a prompting of the Holy Spirit. It is born of the consideration of sin’s ugliness or the fear of eternal damnation and the other penalties threatening the sinner (contrition of fear). Such a stirring of conscience can initiate an interior process which, under the prompting of grace, will be brought to completion by sacramental absolution. By itself however, imperfect contrition cannot obtain the forgiveness of grave sins, but it disposes one to obtain forgiveness in the sacrament of penance (CCC 1452-1453).
However, it is also important to be careful not to become presumptions of God’s mercy and then relax in our habitual sins. God is merciful, however, we must always struggle against our own sins to be in union with God at all times.
I for one knows He answers all prayers and sometimes the answer is no.