Replacing rivets with screws is a common practise. Solid rivets are usually "soft" and work harden as they are driven. Pulled rivets have only a partial wall thickness to offer in shear, when compared to their overall thickness. Most screws have more initial strength, feature a solid shank, and work better to draw two surfaces together when fastened.

If the screws are not tightening and they are run into nuts, chances are they have too long a shank size, or they are not fully threaded to the head. This will prevent the nuts from pulling directly against the surface material, or to tighten against a washer.

With no access to the back side of the screws to see if they are fastened into nuts, or to determine if they are of a sheet metal variety and screw directly into the material, you are limited with alternatives. If you had to remove the metal piece, drilling the heads would be an option, albeit difficult, because the screws could rotate.

If the metal joint in question is secure and not loose, it may be easier to waterproof the area using a sealant tape or elastomeric product. Using the optional reinforcing fabric available for the elastomeric coating, will strenthen the actual joint and reduce the chance of a sealing failure in the future.

If it is a single seam overlap, I would use a marine quality polyurethane caulking/adhesive, either Sikaflex 2XX series or the better 3m 5200 and 4200 series. They far exceed the durability of household silicones and are available in some big box stores or from mail order sources such as Jamestown Dist.