The robots.txt file helps you to determine, which content of your website should be indexed by search engines. Remind that this file is just a kind of recommendation. The search engine might ignore this policy.
Place the robots.txt file in your website root directory.
Example configuration
File: robots.txt
User-agent: *
Disallow: /*.jpg$
Disallow: /*.jpeg$
Disallow: /*.png$
Disallow: /*.gif$
Disallow: /*.pdf$
Disallow: /*.mp4$
Disallow: /*.ico$
Disallow: /imprint
Disallow: /privacy-policy
