|
4cdd726df0
|
Make statistics variable atomic
|
2017-05-01 18:09:40 +02:00 |
|
|
948a796e06
|
Remove crawlinfo
|
2017-04-24 10:36:29 +02:00 |
|
|
e3532f73b4
|
Make SpiderLoop::m_lockTable thread safe
|
2017-04-21 11:50:21 +02:00 |
|
|
a023239b78
|
Make SpiderLoop::m_lockTable private
|
2017-04-18 12:08:52 +02:00 |
|
|
22a931a8aa
|
Move clearLocks logic from SpiderColl to SpiderLoop & change goto to for loop as well
|
2017-04-18 11:49:01 +02:00 |
|
|
299aba3670
|
Remove SpiderLoop::m_activeListModified. No additional usage other than what's already covered by SpiderLoop::m_activeListValid
|
2017-04-12 14:54:12 +02:00 |
|
|
e4855fc1fb
|
Make SpiderLoop::m_activeListValid private.
|
2017-04-12 14:52:32 +02:00 |
|
|
f451b83689
|
Remove SpiderLoop::m_lastCallTime. We're now only calling SpiderLoop::spiderDoledUrls from a single place. Don't need to have logic catering for extra calls.
|
2017-04-12 13:22:14 +02:00 |
|
|
465e41c85d
|
Remove unused SpiderLoop::m_processed
|
2017-04-12 13:17:44 +02:00 |
|
|
c936ea017c
|
Remove redundant keyword class
|
2017-04-12 12:37:15 +02:00 |
|
|
5b87fc7430
|
Rename spiderUrl9 to spiderUrl. Remove SpiderLoop::m_sreq & SpiderLoop::m_doledbKey and instead modify spiderUrl2 to accept those parameters.
|
2017-04-12 12:16:40 +02:00 |
|
|
8cc17fec12
|
Remove SpiderLoop::m_collnum
|
2017-04-12 10:54:23 +02:00 |
|
|
589e94151e
|
Make SpiderLoop::m_maxUsed private
|
2017-04-12 10:11:37 +02:00 |
|
|
7e5bb393cd
|
Remove unused SpiderLoop::m_lockCache
|
2017-04-12 09:48:46 +02:00 |
|
|
01a4f9cca4
|
Make SpiderLoop::m_numSpidersOut private
|
2017-04-11 11:05:33 +02:00 |
|
|
2dc5b02cc6
|
Remove unused printLockTable
|
2017-04-07 11:27:57 +02:00 |
|
|
2d6686ba79
|
Remove unused variable. Make methods/variables private where possible
|
2017-04-06 12:47:47 +02:00 |
|
|
5043a1a22e
|
Make static function part of SpiderLoop class
|
2017-04-06 11:50:57 +02:00 |
|
|
3dbfdf365f
|
Rename SpiderLoop::startLoop to SpiderLoop::init
|
2017-04-05 11:35:04 +02:00 |
|
|
1349b48c41
|
Add locks to RdbTree.
Some code style changes/comment removal.
Moved treetest which was in main.cpp to unittest
|
2017-03-31 12:59:40 +02:00 |
|
|
4b02b5da6c
|
Remove unused variables. Use logDebug instead
|
2017-03-28 14:30:29 +02:00 |
|
|
e80f0290dd
|
Remove unused variable in SpiderLoop
|
2017-03-22 12:31:51 +01:00 |
|
|
f7d0c7e863
|
#include cleanup in SpiderLoop.*
|
2016-11-13 16:12:03 +01:00 |
|
|
aba937780d
|
Stop #including Conf.h from header files
|
2016-11-12 20:24:20 +01:00 |
|
|
1cb65b951f
|
Unused member m_outstanding1 removed from SpiderLoop
|
2016-10-18 22:01:55 +02:00 |
|
|
a040a78c99
|
Use key96_t instead of key_t and redefining std lib key_t (which breaks std lib functionality that uses key_t)
|
2016-09-02 14:49:06 +02:00 |
|
|
fad7c45282
|
Remove Msg12
|
2016-06-29 12:34:23 +02:00 |
|
|
9d632b6578
|
Remove undefined functions
|
2016-03-08 23:14:49 +01:00 |
|
|
ab0b9d03ea
|
Standardize header guards
|
2016-03-08 22:16:02 +01:00 |
|
|
da8734a848
|
More Spider splitting
|
2016-01-30 21:29:32 +01:00 |
|
|
6cee130646
|
Separated Doledb from Spider.cpp
|
2016-01-30 20:11:15 +01:00 |
|
|
8463f36179
|
Split Spider.cpp into Spider, SpiderColl and SpiderLoop. More to come..
|
2016-01-30 19:41:54 +01:00 |
|