# first we address this to all robots User-agent: * # now we specify which directories we don't want the spider to index Disallow: /code Disallow: /images Disallow: /video Disallow: /flash # all done!