0aafb70677
Turn while(1) loops into more conventional for(;;)
2016-08-21 17:35:23 +02:00
f9761e1120
constness
2016-08-19 11:49:02 +02:00
4e4e3371e6
Log function will now return void instead of a boolean
2016-08-01 18:12:10 +02:00
9c92e7d058
Fix conversion from string literal to 'char *' for Unicode
2016-05-31 11:26:01 +02:00
da566cc456
More constness in HttpMime
2016-02-16 11:34:51 +01:00
96b8197ad3
now it compiles with -m32
2014-11-10 14:45:11 -08:00
e7dd8f7956
replace long long with int64_t
2014-10-30 13:36:39 -06:00
f55d4d1230
merge diffbot-testing
2014-04-09 20:10:30 -07:00
9cb99f7621
Merge branch 'diffbot' into diffbot-testing
...
Conflicts:
Spider.cpp
2013-12-16 11:06:11 -08:00
7b768d4b86
Merge branch 'diffbot' into diffbot-testing
...
Conflicts:
iana_charset.cpp
iana_charset.h
2013-12-12 13:01:49 -08:00
16e91375f4
bring in changes from live beta from ~/github.
...
limit spiders to 50, not 500 to prevent oom.
resume killed merges that had num files shrunk even
if down to one file. show collnum in spider queue.
remove back-to-back whitespace, and make all space
a ' ' for getting the doc checksum for deduping.
2013-12-12 12:58:58 -08:00
82494baa89
move CollectionRec stuff into Collectiondb files
...
for simplicity.
2013-12-10 15:28:04 -08:00
02bf6ab3cc
new crawlbot api. not backwards compatible any more.
2013-09-17 10:25:54 -07:00
ca2a024d04
fixed up thread/spider log msgs.
...
fixed core from calling fprintf in
alarm signal missed quickpoll handler.
2013-08-29 21:15:42 -06:00
f6e560c1f4
Initial file population.
2013-08-02 13:12:24 -07:00