Forks/privacore-open-source-search-engine
1
0
Fork 0
forked from Mirrors/privacore-open-source-search-engine
Code Pull Requests Activity

Commit Graph

  • 65bfc3477c Remove commented out code Ai Lin Chia 2017-04-20 14:50:06 +02:00
  • fe0b0c3328 Remove enhancement of error logs (we should always be able to add even while saving - future commit) Ai Lin Chia 2017-04-20 14:33:26 +02:00
  • 52b863c1ef Combine 2 different criteria that is doing the same thing Ai Lin Chia 2017-04-20 14:26:00 +02:00
  • cb3f66fa97 More const Ivan Skytte Jørgensen 2017-04-20 15:55:01 +02:00
  • cf5452c030 Added RdbTree::collExists() just like RdbBuckets Ivan Skytte Jørgensen 2017-04-20 15:14:33 +02:00
  • 12bb0e2acf Introduce Process::isShuttingDown and use it instead of checking Process::EXIT_MODE Ai Lin Chia 2017-04-20 14:00:45 +02:00
  • 9cd74c4046 Merge branch 'master' into nomerge2 Ai Lin Chia 2017-04-19 16:27:10 +02:00
  • a4fd7609d1 Multiply with double instead of int Ai Lin Chia 2017-04-19 16:26:51 +02:00
  • a5543e6273 Merge branch 'master' into nomerge2 Ai Lin Chia 2017-04-19 16:21:24 +02:00
  • 5d94ea73f9 Use fileId instead of fileIdx as fileIdx could change due to merge. Ai Lin Chia 2017-04-19 16:20:24 +02:00
  • f9ace4c96b Wrong variable is used inside for loop. We're always only checking the first file Ai Lin Chia 2017-04-19 16:15:54 +02:00
  • b0b8ace069 Merge branch 'master' into nomerge2 Ai Lin Chia 2017-04-19 15:55:10 +02:00
  • 73a2ed1030 Add max lost positive check Ai Lin Chia 2017-04-19 15:54:15 +02:00
  • a8483620c1 Remove commented out code Ai Lin Chia 2017-04-19 12:48:42 +02:00
  • 89ec1f31f7 Merge branch 'master' into nomerge2 Ai Lin Chia 2017-04-19 12:34:56 +02:00
  • 654d201709 Make Host::m_lastRequestSendTimestamp atomic as there could be multiple threads updating it at the same time Ai Lin Chia 2017-04-19 12:33:34 +02:00
  • 36822bd41d Merge branch 'master' into nomerge2 Ai Lin Chia 2017-04-19 12:14:16 +02:00
  • cc8c66cae3 Remove commented out code Ai Lin Chia 2017-04-19 12:11:36 +02:00
  • 81489c21f9 Merge UdpServer changes unrelated to nomerge feature from nomerge2 branch to master Ai Lin Chia 2017-04-19 12:05:36 +02:00
  • b98590ca0b Make SpiderColl::m_firstKey, SpiderColl::m_nextKey, SpiderColl::m_endKey private Ai Lin Chia 2017-04-18 16:58:32 +02:00
  • dbb69be074 Remove percent calculation for scanning IP. It's only shown on admin/spiderdb, and I almost never see any other number than 0% even when reloading the page multiple times shows the same exact IP. Ai Lin Chia 2017-04-18 16:51:42 +02:00
  • 354e697feb Merge branch 'master' into nomerge2 Ivan Skytte Jørgensen 2017-04-18 16:44:18 +02:00
  • f7fcb2a96c Avoid tagrec lookups on blocked URLs Ivan Skytte Jørgensen 2017-04-18 16:33:59 +02:00
  • 8cf1a67bce Avoid looking up IP of hosts in blocked URLs Ivan Skytte Jørgensen 2017-04-18 16:12:09 +02:00
  • 51f2d3f82c Merge branch 'master' into nomerge2 Ai Lin Chia 2017-04-18 15:36:29 +02:00
  • 02e335cbf6 Add lock to SpiderColl::m_waitingTable Ai Lin Chia 2017-04-18 15:35:42 +02:00
  • b68fdef21d Make SpiderColl::m_waitingTable private Ai Lin Chia 2017-04-18 15:34:16 +02:00
  • a501c66279 Merge branch 'master' into nomerge2 Ai Lin Chia 2017-04-18 15:21:03 +02:00
  • 66598e2714 Add lock to m_doledbIpTable Ai Lin Chia 2017-04-18 14:47:37 +02:00
  • 97ab696a51 Rename m_doleIpTable to m_doledbIpTable & related functions. Move doledbIpTable related functions to be together Ai Lin Chia 2017-04-18 14:37:03 +02:00
  • c60f1e0ad6 Make SpiderColl::m_doleIpTable private Ai Lin Chia 2017-04-18 14:27:19 +02:00
  • 2999c6b6fa Remove always false input variable Ai Lin Chia 2017-04-18 14:16:58 +02:00
  • 97a480a0f6 Merge branch 'master' into nomerge2 Ivan Skytte Jørgensen 2017-04-18 14:12:38 +02:00
  • 53e89c2d15 Revert watchdog functionality. Ivan Skytte Jørgensen 2017-04-18 14:10:45 +02:00
  • 1dbf28bb98 Merge branch 'master' into nomerge2 Ai Lin Chia 2017-04-18 12:38:30 +02:00
  • e4bb72f9f1 Combine ufn check Ai Lin Chia 2017-04-18 12:35:36 +02:00
  • ab34b822b7 Remove unused SpiderColl::m_priorityToUfn Ai Lin Chia 2017-04-18 12:26:13 +02:00
  • 7a8b45e208 Reduce scope of UrlLock declaration Ai Lin Chia 2017-04-18 12:15:17 +02:00
  • b20fc731c5 Use m_lockTable directly instead of going through another variable Ai Lin Chia 2017-04-18 12:09:23 +02:00
  • a023239b78 Make SpiderLoop::m_lockTable private Ai Lin Chia 2017-04-18 12:08:52 +02:00
  • 068cabba0e Use m_lockTable directly instead of going through g_spiderLoop Ai Lin Chia 2017-04-18 11:53:49 +02:00
  • 22a931a8aa Move clearLocks logic from SpiderColl to SpiderLoop & change goto to for loop as well Ai Lin Chia 2017-04-18 11:49:01 +02:00
  • 0f0e92ea0f Fix infinite loop in commit fb1e1ac611 Ai Lin Chia 2017-04-12 16:46:29 +02:00
  • 299aba3670 Remove SpiderLoop::m_activeListModified. No additional usage other than what's already covered by SpiderLoop::m_activeListValid Ai Lin Chia 2017-04-12 14:54:12 +02:00
  • e4855fc1fb Make SpiderLoop::m_activeListValid private. Ai Lin Chia 2017-04-12 14:52:32 +02:00
  • f451b83689 Remove SpiderLoop::m_lastCallTime. We're now only calling SpiderLoop::spiderDoledUrls from a single place. Don't need to have logic catering for extra calls. Ai Lin Chia 2017-04-12 13:22:14 +02:00
  • 465e41c85d Remove unused SpiderLoop::m_processed Ai Lin Chia 2017-04-12 13:17:44 +02:00
  • c936ea017c Remove redundant keyword class Ai Lin Chia 2017-04-12 12:37:15 +02:00
  • 5b87fc7430 Rename spiderUrl9 to spiderUrl. Remove SpiderLoop::m_sreq & SpiderLoop::m_doledbKey and instead modify spiderUrl2 to accept those parameters. Ai Lin Chia 2017-04-12 12:16:40 +02:00
  • 94f391c478 Access m_launches directly instead of through g_spiderLoop Ai Lin Chia 2017-04-12 11:06:51 +02:00
  • 8cc17fec12 Remove SpiderLoop::m_collnum Ai Lin Chia 2017-04-12 10:54:23 +02:00
  • 06e059ddf9 Remove redundant initialization Ai Lin Chia 2017-04-12 10:12:35 +02:00
  • 589e94151e Make SpiderLoop::m_maxUsed private Ai Lin Chia 2017-04-12 10:11:37 +02:00
  • 7e5bb393cd Remove unused SpiderLoop::m_lockCache Ai Lin Chia 2017-04-12 09:48:46 +02:00
  • f70719a36d Remove none useful check. Both are returning true from the function Ai Lin Chia 2017-04-12 09:44:21 +02:00
  • 4018c62c31 Move resetting of g_errno closer to where it is meant to be Ai Lin Chia 2017-04-12 09:43:50 +02:00
  • b64622c628 Code style changes Ai Lin Chia 2017-04-11 22:10:13 +02:00
  • a5077db7a2 Move condition to before lock so we don't lock/getValue unnecessarily Ai Lin Chia 2017-04-11 22:08:11 +02:00
  • 8f7dced649 Remove priority variable that is not used anymore other than for sanity check that is done in scanListForWinners as well (should probably be done in Parms.cpp instead) Ai Lin Chia 2017-04-11 22:03:25 +02:00
  • 5d159f335d Code style changes Ai Lin Chia 2017-04-11 21:58:33 +02:00
  • d098e18d19 Compile variable declaration & assignment into single line Ai Lin Chia 2017-04-11 21:57:01 +02:00
  • 5f1fda3b81 Remove unnecessary boolean. Move logic into if statement Ai Lin Chia 2017-04-11 21:33:50 +02:00
  • 1d62148197 Remove commented out code Ai Lin Chia 2017-04-11 21:26:41 +02:00
  • 226ba6477f Move declaration closer to use Ai Lin Chia 2017-04-11 21:12:31 +02:00
  • a285be1f6d Removed requestPath parameter from urlEncode() (always false) Ivan Skytte Jørgensen 2017-04-11 17:11:43 +02:00
  • 61f8ebef96 Merge branch 'master' into nomerge2 Ivan Skytte Jørgensen 2017-04-11 15:48:52 +02:00
  • 5425f4cd28 Bugfix Dns::getIPLookupHost() Ivan Skytte Jørgensen 2017-04-11 15:48:30 +02:00
  • 46aade7486 Merge branch 'master' into nomerge2 Ivan Skytte Jørgensen 2017-04-11 14:19:31 +02:00
  • 4027028bbd Fix comments where an acient mass-replace had chagned 'int' and 'long' to int32_t in comments Ivan Skytte Jørgensen 2017-04-11 14:18:45 +02:00
  • fb1e1ac611 goto -> for() Ivan Skytte Jørgensen 2017-04-11 14:12:45 +02:00
  • 734c270e78 Merge branch 'master' into nomerge2 Ivan Skytte Jørgensen 2017-04-11 13:36:19 +02:00
  • f69b76bb20 goto -> for() Ivan Skytte Jørgensen 2017-04-11 13:36:05 +02:00
  • 3eaf76cf51 Merge branch 'master' into nomerge2 Ivan Skytte Jørgensen 2017-04-11 12:20:01 +02:00
  • c351607c2e When running out of UDP slots log what there are currently being used for Ivan Skytte Jørgensen 2017-04-11 12:15:57 +02:00
  • 6bfb6fcea9 Remove msg_type_1 which is now unused Ai Lin Chia 2017-04-11 11:09:02 +02:00
  • 01a4f9cca4 Make SpiderLoop::m_numSpidersOut private Ai Lin Chia 2017-04-11 11:05:33 +02:00
  • 9423dd6c5e Merge branch 'master' into nomerge2 Ai Lin Chia 2017-04-11 10:07:26 +02:00
  • 779594a3f9 Remove now redundant public declaration Ai Lin Chia 2017-04-07 16:23:13 +02:00
  • 4e3b41f918 Move private method & variable down Ai Lin Chia 2017-04-07 16:22:23 +02:00
  • e1d45db8b9 Fix segfault when collnum is not 0 Ai Lin Chia 2017-04-11 09:15:21 +02:00
  • d2c1430da8 Fix for crazy high scoring of documents with only a bigram match and no single term matches. Would use default values of 999999999 for minimum single term and pair scores Brian Rasmusson 2017-04-10 20:55:07 +02:00
  • 211b210ccb Merge branch 'master' into nomerge2 Ivan Skytte Jørgensen 2017-04-10 13:51:36 +02:00
  • c86745045d Removed Msg1f / remote log fetching Ivan Skytte Jørgensen 2017-04-10 13:47:33 +02:00
  • 50b656f93a Merge branch 'master' into nomerge2 Ivan Skytte Jørgensen 2017-04-10 12:38:13 +02:00
  • 39eb2890ac Unwrap log line texts for easier grepping Ivan Skytte Jørgensen 2017-04-07 16:00:34 +02:00
  • 85d4284782 Make some HashTableX variables private Ai Lin Chia 2017-04-07 15:54:44 +02:00
  • 493e178cc6 Use HashTableX::set instead of setting m_allocName directly Ai Lin Chia 2017-04-07 14:58:43 +02:00
  • 858373310e Use isInitialized instead of checking m_ks directly Ai Lin Chia 2017-04-07 14:36:29 +02:00
  • f9c1a65c6b Access using functions instead of direct variable access Ai Lin Chia 2017-04-07 14:35:46 +02:00
  • 4128fe14ea Use getNumSlots instead of m_numSlots Ai Lin Chia 2017-04-07 14:19:07 +02:00
  • f13e18b5c6 Set HashTableX::m_maskKeyOffset in set function instead of accessing it directly Ai Lin Chia 2017-04-07 14:18:04 +02:00
  • 0f54c3b76b Assign variable directly Ai Lin Chia 2017-04-07 14:13:27 +02:00
  • a1e990508a Use isInitialized function instead of checking m_ks Ai Lin Chia 2017-04-07 14:13:04 +02:00
  • 9e44fe235f Replace direct variable use with getNumUsedSlots() Ai Lin Chia 2017-04-07 14:12:46 +02:00
  • 16267e9ba0 Add isWritable() method for HashTableX and make m_isWritable private Ai Lin Chia 2017-04-07 13:08:54 +02:00
  • d895ca7358 Set HashTableX::m_useKeyMagic in constructor instead Ai Lin Chia 2017-04-07 13:06:56 +02:00
  • 3768d23353 Rename getNumSlotsUsed to getNumUsedSlots to keep things consistent Ai Lin Chia 2017-04-07 13:05:01 +02:00
  • 4ba0c00657 Replace use of HashTableX::getNumSlotsUsed() with HashTableX::getNumUsedSlots() Ai Lin Chia 2017-04-07 13:03:05 +02:00
  • 98e87a100a Replace HashTableX::m_numSlots usage with HashTableX::isInitialized & HashTableX::getNumSlots where applicable. Simplify HashTableX::isEmpty Ai Lin Chia 2017-04-07 12:50:22 +02:00
  • c7b334049a Remove commented out code Ai Lin Chia 2017-04-07 12:07:12 +02:00