Yellow, orange, or red filter does NOT give you more contrast.... perhaps it is sometimes called that but all it does is to change the relationship between different natural colors - which might or might not result in increase in contrast. If you are shooting a scene that includes cloud, yes it does because the blue sky gets darker in comparison to the white cloud.
Looking at the photographs you gave as examples, I'd say no, don't use filters. Also, when one uses filter, he/she has a goal. I want THIS color to be darker, so I pick THIS filter. Putting one on neither (always) increase your contrast, nor improve your photograph.
If you use RED filter on your photograph with bricks, it will do funny things. Red will cause your brick (which is usually reddish) to be lighter because it will pass more red light which in turn will make that portion of your negative darker. Which in turn make your prints lighter, which in comparison with white sign and grout within the brick wall, it will be LESS contrast.