privacore-open-source-search-engine/robots at master - privacore-open-source-search-engine - Code @ MoldyBits.net

Mirrors/privacore-open-source-search-engine

mirror of https://github.com/privacore/open-source-search-engine.git synced 2025-06-04 21:09:33 -04:00

History

Ai Lin Chia 3bf9f64109 Fix valgrind invalid read error on Robots parsing

2016-05-24 12:06:43 +02:00

..

boe.es

Add more unit test for RobotRule & unit test for real robots.txt found in the wild

2016-04-05 23:21:50 +02:00

coriolis.io

Fix valgrind invalid read error on Robots parsing

2016-05-24 12:06:43 +02:00

needrom.com

Add more unit test for RobotRule & unit test for real robots.txt found in the wild

2016-04-05 23:21:50 +02:00

reddit.com

Add more unit test for RobotRule & unit test for real robots.txt found in the wild

2016-04-05 23:21:50 +02:00

speedtest.net

Add more unit test for RobotRule & unit test for real robots.txt found in the wild

2016-04-05 23:21:50 +02:00

state.gov

Add more unit test for RobotRule & unit test for real robots.txt found in the wild

2016-04-05 23:21:50 +02:00

thekitchn.com

Add more unit test for RobotRule & unit test for real robots.txt found in the wild

2016-04-05 23:21:50 +02:00

upf.edu

Fix valgrind invalid read error on Robots parsing

2016-05-24 12:06:43 +02:00

wikipedia.org

Add more unit test for RobotRule & unit test for real robots.txt found in the wild

2016-04-05 23:21:50 +02:00