Forks/privacore-open-source-search-engine
1
0
Fork 0
forked from Mirrors/privacore-open-source-search-engine
Code Pull Requests Activity

Commit Graph

  • 2a57cab2d5 #include cleanup Ivan Skytte Jørgensen 2016-07-11 14:03:29 +02:00
  • 24657cd424 Fix memleak on BigFileTest Ai Lin Chia 2016-07-11 13:54:32 +02:00
  • 19831bee22 Code style changes Ai Lin Chia 2016-07-11 13:46:37 +02:00
  • 93328cbc71 Code style changes Ai Lin Chia 2016-07-08 17:21:19 +02:00
  • 4dccc715bd #include cleanup Ivan Skytte Jørgensen 2016-07-11 13:53:33 +02:00
  • 4b5da95eaa #include cleanup Ivan Skytte Jørgensen 2016-07-11 12:54:17 +02:00
  • 478f41360b #include cleanup Ivan Skytte Jørgensen 2016-07-11 12:51:11 +02:00
  • 8b3a025688 rename RdbScan::m_list to RdbScan::m_rdblist. Don't like the use of m_list->m_list of different types Brian Rasmusson 2016-07-11 12:54:54 +02:00
  • d9b0e10335 #include cleanup Ivan Skytte Jørgensen 2016-07-11 12:48:53 +02:00
  • 8d54c24991 Close of more of Msg39 Ivan Skytte Jørgensen 2016-07-11 12:36:29 +02:00
  • d634c7bff4 Renamed Msg39::m_tmpq to m_query Ivan Skytte Jørgensen 2016-07-11 12:31:03 +02:00
  • acd685186a core detection and alerting fixes Brian Rasmusson 2016-07-09 14:22:46 +02:00
  • da4e1b8522 only alert if core is from gb Brian Rasmusson 2016-07-09 12:05:40 +02:00
  • e2a0585d93 improved alerting when core is detected. now includes and backs up stack trace Brian Rasmusson 2016-07-09 11:16:11 +02:00
  • 670be7394b Simplify Msg39::controlLoop() a bit Ivan Skytte Jørgensen 2016-07-08 17:10:34 +02:00
  • df8e9f9d17 Remove code to clean out existing spiderdb records that should not be inserted anymore. Spiderdb on all shards have been merged. Ai Lin Chia 2016-07-08 15:51:40 +02:00
  • 9713886a40 Fix a bug where when a killed merge is resumed and only renaming is needed, the next new file created is not updated to the next id. Ai Lin Chia 2016-07-08 11:54:23 +02:00
  • cf2c6b165a Code style changes. Change return log() to log(); return false; Ai Lin Chia 2016-07-07 10:29:05 +02:00
  • dfdba8cbb3 Code style changes Ai Lin Chia 2016-07-06 17:51:01 +02:00
  • 45ed003ed3 added first attempt of alerting on error Brian Rasmusson 2016-07-07 21:38:38 +02:00
  • cbb966b684 made Msg2:.m_minRecSizes const Ivan Skytte Jørgensen 2016-07-07 17:31:55 +02:00
  • 5d8f3f9be7 Decouple Msg2 from Msg39 Ivan Skytte Jørgensen 2016-07-07 16:16:16 +02:00
  • 0865ef7f09 Removed superfluous Msg39::m_docIdStart/m_docIdEnd Ivan Skytte Jørgensen 2016-07-07 16:02:39 +02:00
  • d4d23111a8 Merge branch 'master' of github.com:privacore/open-source-search-engine Ivan Skytte Jørgensen 2016-07-07 14:19:52 +02:00
  • 25726ca7d9 More symbol hiding Ivan Skytte Jørgensen 2016-07-07 14:14:58 +02:00
  • de5f2a7a8d Removed comment no longer applicable Ivan Skytte Jørgensen 2016-07-07 13:53:15 +02:00
  • 798ca0ffad Fixed out-of-date comment Ivan Skytte Jørgensen 2016-07-07 13:52:13 +02:00
  • 6110e68f90 Remvoed unused Msg5::m_cacheKey Ivan Skytte Jørgensen 2016-07-07 13:36:59 +02:00
  • a9348ee81e Removed write-only member Msg5::m_prevCount Ivan Skytte Jørgensen 2016-07-07 13:35:56 +02:00
  • febb10062e Remvoed Msg5::m_parent which was a hack Ivan Skytte Jørgensen 2016-07-07 13:33:03 +02:00
  • c2639d32e2 Fix compilation Ivan Skytte Jørgensen 2016-07-07 12:06:15 +02:00
  • 764bc78cd3 Better encapsulation of Msg2 Ivan Skytte Jørgensen 2016-07-07 11:57:23 +02:00
  • 7189f27808 Rename BigFile::chopHead to BigFile::unlinkPart Ai Lin Chia 2016-07-06 16:51:20 +02:00
  • f318437729 Code style changes Ai Lin Chia 2016-07-06 16:50:17 +02:00
  • 02f722166f Code style changes. Log level changes. Ai Lin Chia 2016-07-06 14:55:13 +02:00
  • 1439ba1816 Code style changes Ai Lin Chia 2016-07-06 11:13:52 +02:00
  • 8ce4a80c86 Exit from look if fatal_error file is found Ai Lin Chia 2016-07-06 11:12:23 +02:00
  • f86e945d49 Add force/unforced rename and fix bug where first file was renamed Ai Lin Chia 2016-07-05 16:50:45 +02:00
  • 04664503fb Reduced coupling to Msg2::getLists() Ivan Skytte Jørgensen 2016-07-05 16:33:43 +02:00
  • f42fba690c fixed core in RdbBuckets::fastLoadColl when resizing m_buckets during posdb rebuild Brian Rasmusson 2016-07-05 16:30:18 +02:00
  • 6994859805 Introduced gbshutdownLogicError() and gbshutdownCorrupted() Ivan Skytte Jørgensen 2016-07-05 15:14:07 +02:00
  • baddd9e74e Made Posdb::getDocId() faster Ivan Skytte Jørgensen 2016-07-05 14:03:09 +02:00
  • 9de8cb33aa Clean up Posdb*.h to remove duplicated symbols Ivan Skytte Jørgensen 2016-07-05 12:54:26 +02:00
  • 4d01557b00 Fix bug when handling extra long url (url longer than MAX_URL_LEN) Ai Lin Chia 2016-07-05 12:18:54 +02:00
  • d4094f7dcd help flexelint Ivan Skytte Jørgensen 2016-07-05 12:18:50 +02:00
  • a6af616b6e Use gbshutdownAbort() instead of g_process.shutdownAbort() Ivan Skytte Jørgensen 2016-07-05 12:11:14 +02:00
  • d9b76ff04c optimize (-O2) PosdbTable.cpp Ivan Skytte Jørgensen 2016-07-05 12:01:41 +02:00
  • 8927f230d6 Revert "Simplify getWordPosList() a bit" Ivan Skytte Jørgensen 2016-07-05 12:01:11 +02:00
  • ee129052d1 Simplify getWordPosList() a bit Ivan Skytte Jørgensen 2016-07-04 15:50:21 +02:00
  • 86ad936a25 added constructor so Url objects are correctly initialized when a set method has not been called Brian Rasmusson 2016-07-05 11:02:52 +02:00
  • bab0b32b26 Dump list of files at every gb run Ai Lin Chia 2016-07-04 17:09:56 +02:00
  • ac790137a8 Disable log trace for unit test Ai Lin Chia 2016-07-04 16:15:36 +02:00
  • 98e3284b7a Change some debug logs to err/warn Ai Lin Chia 2016-07-04 16:11:38 +02:00
  • c5cc33ab42 Check for file existence before renaming to avoid removing files Ai Lin Chia 2016-07-04 16:10:00 +02:00
  • de2b46a09e Split Posdb.* source files Ivan Skytte Jørgensen 2016-07-04 15:06:15 +02:00
  • 17590c21b4 Add EDOCFORCEDELETE to be part of deleted count Ai Lin Chia 2016-07-04 12:13:37 +02:00
  • 4c9048236f Add gb spider statistics Ai Lin Chia 2016-07-04 11:26:37 +02:00
  • 36c7faa05d Replace usage of mstrerrno with mstrerror. Remove mstrerrno. Add merrname for mapping of errno to error name. Ai Lin Chia 2016-07-04 11:20:34 +02:00
  • ab1e3836e7 Removed silly pointer underflow check Ivan Skytte Jørgensen 2016-07-01 13:54:09 +02:00
  • b87804c05b Removed unnecessary null check Ivan Skytte Jørgensen 2016-07-01 12:35:11 +02:00
  • 55e93de571 Fix null pointer dereference Ivan Skytte Jørgensen 2016-07-01 12:33:30 +02:00
  • 8fe2aebf05 Remove commented out code Ai Lin Chia 2016-06-30 11:06:13 +02:00
  • 40412c5d45 Use EMFILE instead of 24 & E2BIG instead of 7 Ai Lin Chia 2016-06-29 17:16:53 +02:00
  • 45f88eb8e5 Remove EHITPROCESSLIMIT. g_errno is never set to EHITPROCESSLIMIT Ai Lin Chia 2016-06-29 16:55:49 +02:00
  • afdd437956 Remove EHITCRAWLLIMIT. g_errno is never set to EHITCRAWLLIMIT Ai Lin Chia 2016-06-29 16:54:18 +02:00
  • eb289701e6 Remove EINLINESECTIONS. g_errno is never set to EINLINESECTIONS Ai Lin Chia 2016-06-29 16:51:14 +02:00
  • d6da804e5d Remove some EDIFFBOT* errors. g_errno is never set to them Ai Lin Chia 2016-06-29 16:48:50 +02:00
  • 3227f38b27 Remove ETOOEARLY. g_errno is never set to ETOOEARLY Ai Lin Chia 2016-06-29 16:44:07 +02:00
  • aa65bcbe26 Modify hardcoded g_errno value 7 to E2BIG Ai Lin Chia 2016-06-29 16:40:40 +02:00
  • f982c35079 Remove EURLHASNOIP. g_errno is never set to EURLHASNOIP Ai Lin Chia 2016-06-29 16:39:51 +02:00
  • 6d39517728 Remove ETOOMANYOPERANDS. g_errno is never set to ETOOMANYOPERANDS Ai Lin Chia 2016-06-29 16:34:46 +02:00
  • fb896ac661 Remove EMISSINGQUERYTERMS. g_errno is never set to EMISSINGQUERYTERMS Ai Lin Chia 2016-06-29 16:32:04 +02:00
  • dc0e92cdc0 Remove unused errno Ai Lin Chia 2016-06-29 16:27:18 +02:00
  • fad7c45282 Remove Msg12 Ai Lin Chia 2016-06-29 12:34:23 +02:00
  • 76dc08637f Code style changes Ai Lin Chia 2016-06-29 12:05:02 +02:00
  • 51b74f69ff Add UrlParserTest for scheme & relative scheme Ai Lin Chia 2016-06-29 11:36:34 +02:00
  • 48856025db Code style changes Ai Lin Chia 2016-06-29 11:34:06 +02:00
  • 6349ac3808 Remove unused Stats::dumpGIF Ai Lin Chia 2016-06-29 11:32:40 +02:00
  • f1bb90513a Code style changes Ai Lin Chia 2016-06-29 11:28:58 +02:00
  • ff71698d97 Partially fix "return log(...);" to "log(...); return false;" Ai Lin Chia 2016-06-29 11:14:39 +02:00
  • 596aa0ded5 Handle empty strings in getTLD() Ivan Skytte Jørgensen 2016-06-28 16:46:36 +02:00
  • 5fa1e4515c another NULL-check in warc import code Brian Rasmusson 2016-06-28 16:05:56 +02:00
  • 7f86968533 Fix valgrind uninitialized data warning Ai Lin Chia 2016-06-28 15:49:05 +02:00
  • 608b025905 Fix pointer overflows Ivan Skytte Jørgensen 2016-06-28 15:14:53 +02:00
  • 05b5f63ffb Fix const-string Ivan Skytte Jørgensen 2016-06-28 14:45:08 +02:00
  • 8c4f30cba6 Fix const-string Ivan Skytte Jørgensen 2016-06-28 14:39:41 +02:00
  • 577107ab04 Added comment about LINKDBEPOCH Ivan Skytte Jørgensen 2016-06-28 14:22:44 +02:00
  • 14591acae7 Moved up NULL-check to before first use in XmlDoc::set2 Brian Rasmusson 2016-06-28 14:12:53 +02:00
  • 6592d61b9b Merge branch 'master' of https://github.com/privacore/open-source-search-engine Brian Rasmusson 2016-06-28 14:02:10 +02:00
  • aaf262b2e6 NULL-check in gotReplyWrapperP and in doInjectWarc Brian Rasmusson 2016-06-28 14:02:04 +02:00
  • 5c50262bbd Add more logging on git commit id (so we can match logs with binary) Ai Lin Chia 2016-06-28 12:26:26 +02:00
  • 0a65d2ac9a NULL-check in StateCD::printSpiderdbList Brian Rasmusson 2016-06-28 12:03:51 +02:00
  • 63b4b38816 Fix addMatches segfault Ai Lin Chia 2016-06-28 11:48:06 +02:00
  • 1740b9af00 Change tagdb maxtreesize for test version to enable easier testing Ai Lin Chia 2016-06-28 11:07:03 +02:00
  • bd9ff20d78 Fix unreachable-code warnings Ai Lin Chia 2016-06-28 11:01:47 +02:00
  • 7615174151 NULL-check in SpiderLoop::spiderDoledUrls Brian Rasmusson 2016-06-28 11:42:24 +02:00
  • 768a6509e5 Added NULL-check in HttpRequest::set Brian Rasmusson 2016-06-27 21:05:37 +02:00
  • c507afbcf7 added NULL-check and fixed return value in Proxy::handleRequest Brian Rasmusson 2016-06-27 20:26:37 +02:00
  • 4474a1d5b2 corrected NULL-check in PosdbTable::init Brian Rasmusson 2016-06-27 20:14:01 +02:00
  • 3f7ce4b9a6 Merge branch 'master' of github.com:privacore/open-source-search-engine Ivan Skytte Jørgensen 2016-06-27 16:47:08 +02:00