Forks/privacore-open-source-search-engine
1
0
Fork 0
forked from Mirrors/privacore-open-source-search-engine
Code Pull Requests Activity

Commit Graph

  • 18bb35eeb2 Removed configuration items for sending email (was handled by PingServer) Ivan Skytte Jørgensen 2017-05-07 14:24:42 +02:00
  • 3415d008ba Removed configuration associated with PingServer Ivan Skytte Jørgensen 2017-05-07 14:18:35 +02:00
  • b38a4f2b82 Cleaned out commented-out code in pagehosts.cpp Ivan Skytte Jørgensen 2017-05-05 15:33:21 +02:00
  • 9e3d69fb9e Clearly indicate 'this' host in hostlist in pageHosts.cpp Ivan Skytte Jørgensen 2017-05-05 15:24:01 +02:00
  • 1c117d3aa1 Dropped PingInfo + PingServer entierely. Ivan Skytte Jørgensen 2017-05-05 15:00:31 +02:00
  • cfd1b838b2 Removed write-only variable Host::m_gotPingReply Ivan Skytte Jørgensen 2017-05-05 14:18:20 +02:00
  • 5cddcfeece dailymerge: use HostRuntimeInformation::m_valid instead of gotpingreply Ivan Skytte Jørgensen 2017-05-05 14:16:21 +02:00
  • 4f124833e9 pagehosts: use HostRuntimeInformation::m_valid instead of crc check Ivan Skytte Jørgensen 2017-05-05 14:15:02 +02:00
  • 5790b7f7db Renamed HostRuntimeInformation::m_flagsValid to m_valid Ivan Skytte Jørgensen 2017-05-05 14:13:59 +02:00
  • 8169ce0561 Add gb_output*.txt to gitignore Ai Lin Chia 2017-05-05 14:06:35 +02:00
  • edb2423c5a moved PingInfo::m_repairMode and logic form PingServer to Hostdb+Repair.cpp Ivan Skytte Jørgensen 2017-05-05 14:04:52 +02:00
  • d2643169de Add encoding statistics so we can keep track of what we detect as encoding, and what language we detect it as Ai Lin Chia 2017-05-05 12:39:51 +02:00
  • 660cf3ca55 Removed unused PingServer::getMaxRepairMode() Ivan Skytte Jørgensen 2017-05-05 12:29:39 +02:00
  • cce5deebda In a single-instance setup hostsconfCRC is always consistent (even more bugfixed now) Ivan Skytte Jørgensen 2017-05-05 12:14:22 +02:00
  • f65b3a8529 In a single-instance setup hostsconfCRC is always consistent Ivan Skytte Jørgensen 2017-05-05 12:12:56 +02:00
  • de1a28f958 Dont pthread_join if vagus is disabled Ivan Skytte Jørgensen 2017-05-05 12:10:15 +02:00
  • 1aa35b48f0 bugfix vagus cluster name (forgot m_obj) Ivan Skytte Jørgensen 2017-05-05 12:06:17 +02:00
  • 7e7db15f5d Moved hosts.conf CRC info and check from PingServer to Hostdb+vagus Ivan Skytte Jørgensen 2017-05-04 17:57:57 +02:00
  • 4a2d785ad1 Add more logs so we can see where we're getting the encoding for the document Ai Lin Chia 2017-05-04 16:54:19 +02:00
  • fcac36be0f Show 'N/A' for ping times until we have ping'ed a host Ivan Skytte Jørgensen 2017-05-04 16:39:24 +02:00
  • e6d7f1a401 Moved PingINfo::m_totalDocsIndexed to Host::RuntimeHostInformation Ivan Skytte Jørgensen 2017-05-04 16:18:39 +02:00
  • 63ed7b6351 Moved some fields originating from PingInfo from Host to HostRuntimeInformation Ivan Skytte Jørgensen 2017-05-04 15:58:58 +02:00
  • ee6b6657ab Added Host::isHostsConfCRCKnown() Ivan Skytte Jørgensen 2017-05-04 15:41:14 +02:00
  • 26f4d7b1be Make RdbMap::m_numPositiveRecs & RdbMap::m_numNegativeRecs atomic Ai Lin Chia 2017-05-04 15:42:59 +02:00
  • 9388fac139 Don't try to merge from dumpTree thread. Let the usual callback trigger the merge (hardcoded to every 60s) Ai Lin Chia 2017-05-04 15:33:43 +02:00
  • 6705dc21c1 Made Host::hasSameHostsConfCRC() method Ivan Skytte Jørgensen 2017-05-04 15:16:50 +02:00
  • ed01207939 Try to set langId from charset if it's still unknown by the end of the various detection we use Ai Lin Chia 2017-05-04 15:14:47 +02:00
  • 4d7aeb221a Moved PingInfo::m_gbVersionStr to Host:: Ivan Skytte Jørgensen 2017-05-04 15:01:55 +02:00
  • aab478e6d6 Moved PingInfo::m_dailyMergeCollnum to Host:: Ivan Skytte Jørgensen 2017-05-04 14:53:23 +02:00
  • 0a851bc88e Added comment on what m_unused8 used to be Ivan Skytte Jørgensen 2017-05-04 14:42:22 +02:00
  • af8c72dfb6 Remvoed descriptions in PageHosts.cpp which no longer exist Ivan Skytte Jørgensen 2017-05-04 14:42:02 +02:00
  • 61dc9e4573 Keep our own Host::m_flags up-to-date Ivan Skytte Jørgensen 2017-05-04 14:37:19 +02:00
  • 5a78eac74f Moved PingInfo::m_flags to Host Ivan Skytte Jørgensen 2017-05-04 14:01:10 +02:00
  • 4734e77293 Make some logic more straight-forward in pageHosts.cpp Ivan Skytte Jørgensen 2017-05-04 13:38:02 +02:00
  • 6097e3b8d3 Untangle some messes in PingServer Ivan Skytte Jørgensen 2017-05-04 13:04:10 +02:00
  • 8f19577d8f Use Vagus for detecting dead instances Ivan Skytte Jørgensen 2017-05-04 12:06:19 +02:00
  • f6b209f1ca Don't talk to Vagus if only 1 host is configured i hosts.conf Ivan Skytte Jørgensen 2017-05-04 11:48:34 +02:00
  • 52191466a0 Use std::atomic for Rdb::m_isDumping Ai Lin Chia 2017-05-03 10:14:24 +02:00
  • 5ff827ac26 Merge branch 'master' into dev-dumpthread Ai Lin Chia 2017-05-03 10:11:57 +02:00
  • 9a2f5580c5 Add lock for accessing m_fileInfo array Ai Lin Chia 2017-05-03 10:10:54 +02:00
  • 92e16fdfcc Removed commented out code Ai Lin Chia 2017-05-02 17:12:42 +02:00
  • 811180093d Merge branch 'master' into dev-dumpthread Ai Lin Chia 2017-05-02 13:05:15 +02:00
  • 1ac82c163a More fixes Ai Lin Chia 2017-05-02 13:04:38 +02:00
  • 9650f57481 PingServer: more const Ivan Skytte Jørgensen 2017-05-02 13:03:39 +02:00
  • 4275ee54e2 Dropped Hostdb 'net name' - it was always 'default' Ivan Skytte Jørgensen 2017-05-02 12:59:41 +02:00
  • a829241d1a Hostdb: more constness Ivan Skytte Jørgensen 2017-05-02 12:55:34 +02:00
  • 97883053f4 Merge branch 'master' into dev-dumpthread Ai Lin Chia 2017-05-02 13:00:28 +02:00
  • ad4044482e More fixes around using unlocked version instead of locked (for debug log) Ai Lin Chia 2017-05-02 13:00:03 +02:00
  • 856a71e7d3 Merge branch 'master' into dev-dumpthread Ai Lin Chia 2017-05-02 12:55:21 +02:00
  • 2fc04dfe9d Fix abort when debug log is enabled. Call locked version instead of unlocked version. Ai Lin Chia 2017-05-02 12:53:07 +02:00
  • 201594f987 Removed PingServer::m_numHostsWithForeignRecs Ivan Skytte Jørgensen 2017-05-02 12:42:51 +02:00
  • b7fef6c9b5 Moved dead host counting logic from PingServer to Hostdb Ivan Skytte Jørgensen 2017-05-02 12:41:03 +02:00
  • 49ce430809 Merge branch 'master' into dev-dumpthread Ai Lin Chia 2017-05-02 12:22:13 +02:00
  • 5343035fd7 Back off reconnecting to Vagus Ivan Skytte Jørgensen 2017-05-02 12:13:57 +02:00
  • ec7d909a49 Removed Host::m_firstOOMTime Ivan Skytte Jørgensen 2017-05-02 12:03:47 +02:00
  • 9143000249 Removed Host::m_loadAvg Ivan Skytte Jørgensen 2017-05-02 12:02:58 +02:00
  • 0e1120eec7 Removed parameter from InstanceInfoExchange::weAreAlive() Ivan Skytte Jørgensen 2017-05-02 11:57:22 +02:00
  • 66671085b3 Merge branch 'master' into dev-dumpthread Ai Lin Chia 2017-05-02 11:40:53 +02:00
  • 3200de77f0 More changes. Pipe gb output to a separate file Ai Lin Chia 2017-05-02 11:40:22 +02:00
  • 95524f631a Merge branch 'master' into dev-dumpthread Ai Lin Chia 2017-05-02 11:26:10 +02:00
  • 7d0c3889ec More gbstart changes Ai Lin Chia 2017-05-02 11:25:48 +02:00
  • 08020e0a38 Merge branch 'master' into dev-dumpthread Ai Lin Chia 2017-05-02 11:10:01 +02:00
  • ad260b49db Let gbstart script move logfile instead of gb process. This is because if we start piping into existing log file, we're not reopening the fd when gb moves it Ai Lin Chia 2017-05-02 11:09:10 +02:00
  • 75fecf8a32 Merge branch 'master' into dev-dumpthread Ai Lin Chia 2017-05-02 10:59:16 +02:00
  • 326f9bafb4 Pipe gbstart output to logfile Ai Lin Chia 2017-05-02 10:46:18 +02:00
  • 643fe2de40 Merge branch 'master' into dev-dumpthread Ai Lin Chia 2017-05-01 18:13:00 +02:00
  • 4cdd726df0 Make statistics variable atomic Ai Lin Chia 2017-05-01 18:09:27 +02:00
  • abac6204b4 Use separate keystr buffer Ai Lin Chia 2017-05-01 18:08:08 +02:00
  • d3ecbda478 Use Vagus for some host/instnace information Ivan Skytte Jørgensen 2017-05-01 17:21:34 +02:00
  • 567032b716 Merge branch 'master' into dev-dumpthread Ai Lin Chia 2017-05-01 17:03:11 +02:00
  • 92620ffa3b Merge branch 'master' into nomerge2 Ai Lin Chia 2017-05-01 16:52:33 +02:00
  • 01887a2d92 Filter out simplified redirection result Ai Lin Chia 2017-05-01 16:50:17 +02:00
  • 320d62a002 Clear content ptr if it's non-canonical/simplified redirect error Ai Lin Chia 2017-05-01 16:34:15 +02:00
  • 3d59d15ae6 Extend check to check for less than 0 as well Ai Lin Chia 2017-05-01 16:33:51 +02:00
  • 1d029f903f Show http status on page info Ai Lin Chia 2017-05-01 16:18:31 +02:00
  • 00e9e43345 Removed now-dead code in Pages.cpp Ivan Skytte Jørgensen 2017-05-01 16:41:06 +02:00
  • 81eda2d740 Merge branch 'master' of github.com:privacore/open-source-search-engine Ivan Skytte Jørgensen 2017-05-01 15:58:46 +02:00
  • 5f40fca803 swap(IOBuffer,IOBuffer) should be inline Ivan Skytte Jørgensen 2017-05-01 15:58:37 +02:00
  • 90768196e1 Revert "Minor UI fixes. Don't print initial dash before scoring if cache link is not shown" Ai Lin Chia 2017-05-01 15:27:32 +02:00
  • 980d6cc427 Merge branch 'master' into nomerge2 Ai Lin Chia 2017-05-01 15:07:37 +02:00
  • f55ba149d4 Revert "Let's hash content type for xml/json document as well" Ai Lin Chia 2017-05-01 15:06:34 +02:00
  • 6fc926e67d Merge branch 'master' into dev-dumpthread Ai Lin Chia 2017-05-01 14:45:20 +02:00
  • 1b0131b920 Merge remote-tracking branch 'origin/master' into nomerge2 Brian Rasmusson 2017-05-01 14:45:08 +02:00
  • 81f3fb8234 Let's hash content type for xml/json document as well Ai Lin Chia 2017-05-01 14:36:35 +02:00
  • 0b9b25f410 Filter out empty simplified redirection/non-canonical pages Ai Lin Chia 2017-05-01 14:19:52 +02:00
  • ea05b7388b Fix Msg20Reply::m_contentLen. Size is always +1 of length Ai Lin Chia 2017-05-01 14:15:42 +02:00
  • cf05aa2149 Minor UI fixes. Don't print initial dash before scoring if cache link is not shown Ai Lin Chia 2017-05-01 14:15:21 +02:00
  • 28a942905f Make sure posdb generated while rebuilding is the same as spidering Ai Lin Chia 2017-05-01 13:58:28 +02:00
  • 9b930818a0 Use size_utf8Content instead of m_contentLen Ai Lin Chia 2017-05-01 12:25:06 +02:00
  • 836da2ae32 Don't index body/inlink text when we have no content Ai Lin Chia 2017-05-01 12:06:46 +02:00
  • 4c860a94a4 Don't hash gbcontenthash when contentLen is 0 Ai Lin Chia 2017-05-01 12:04:36 +02:00
  • daf99a093e Remove temp variable Ai Lin Chia 2017-05-01 12:02:47 +02:00
  • fc2ca29591 Don't index mid domain url for document with no content Ai Lin Chia 2017-05-01 11:52:11 +02:00
  • cf79f0c670 Remove now unused code (we have an early return for json/xml content type) Ai Lin Chia 2017-05-01 11:51:04 +02:00
  • e9003b1540 Remove commented out code Ai Lin Chia 2017-05-01 11:33:48 +02:00
  • 71b32ddf3c Code style changes Ai Lin Chia 2017-05-01 11:30:27 +02:00
  • d257d74cf0 Remove unused input variables Ai Lin Chia 2017-05-01 11:20:30 +02:00
  • 9a54b7de4d Remove commented out code Ai Lin Chia 2017-05-01 11:03:45 +02:00
  • fc88c4966b Merge remote-tracking branch 'origin/master' into nomerge2 Brian Rasmusson 2017-05-01 14:06:11 +02:00
  • 8f992ee6cd getLocalIps(): skip non-IPv4 correctly Ivan Skytte Jørgensen 2017-05-01 13:43:54 +02:00