Ai Lin Chia 3bf9f64109 Fix valgrind invalid read error on Robots parsing

2016-05-24 12:06:43 +02:00

..

boe.es

Add more unit test for RobotRule & unit test for real robots.txt found in the wild

2016-04-05 23:21:50 +02:00

coriolis.io

Fix valgrind invalid read error on Robots parsing

2016-05-24 12:06:43 +02:00

needrom.com

Add more unit test for RobotRule & unit test for real robots.txt found in the wild

2016-04-05 23:21:50 +02:00

reddit.com

Add more unit test for RobotRule & unit test for real robots.txt found in the wild

2016-04-05 23:21:50 +02:00

speedtest.net

Add more unit test for RobotRule & unit test for real robots.txt found in the wild

2016-04-05 23:21:50 +02:00

state.gov

Add more unit test for RobotRule & unit test for real robots.txt found in the wild

2016-04-05 23:21:50 +02:00

thekitchn.com

Add more unit test for RobotRule & unit test for real robots.txt found in the wild

2016-04-05 23:21:50 +02:00

upf.edu

Fix valgrind invalid read error on Robots parsing

2016-05-24 12:06:43 +02:00

wikipedia.org

Add more unit test for RobotRule & unit test for real robots.txt found in the wild

2016-04-05 23:21:50 +02:00