Modesty is NOT about "not drawing attention to yourself", if this was the case then Christians should be walking around half naked like the rest of the world. Part of modesty is about NOT drawing attention to yourself for ungodly reasons. If someone is starring at you because you are dressed half naked, wearing gaudy or flashy clothing THIS goes against the Word of God. If someone is starring at you because you are dressed plainly and modestly then you ARE still following the Word of God by being set apart and not being conformed to the world. We are to be set apart we are NOT to look like the world, let us look the Christians NOT the World. WE should seek to follow the Word of God not worry about looking different or standing out. Stand out for Christ and ignore the people who stare and mock, they are not your leader. God is.
Do not be conformed to this world, but be transformed by the renewal of your mind, that by testing you may discern what is the will of God, what is good and acceptable and perfect.