Lots of things you can do and it depends on the situation, so images of your exact problem are best general things to try: find a larger image to use for mask use a noise brush to paint the mask manually use an upscaler (free on internet) to increase resolution of an image use geometry to screen view of large areas layer…
For macro scale variation on things like rocks it is pretty common to use world space projected textures. that's done in the shader in the engine. Images would help a lot. Words like "large" and "better quality" aren't very precise.