Forks/privacore-open-source-search-engine
1
0
Fork 0
forked from Mirrors/privacore-open-source-search-engine
Code Pull Requests Activity

Commit Graph

  • b18e363e3e valgrind: read memory below stack Ivan Skytte Jørgensen 2016-01-12 15:01:14 +01:00
  • 8960ab655e Made botname checked in metatags and robots.txt configurable Brian Rasmusson 2016-01-12 14:16:04 +01:00
  • 16a7796ae8 Made botname checked in metatags and robots.txt configurable Brian Rasmusson 2016-01-12 14:14:02 +01:00
  • c6cccb77db m_nodes[] were not always assigned values causing valgrind to (rightfully) complain Ivan Skytte Jørgensen 2016-01-12 13:52:05 +01:00
  • c5897ef121 More valgrind suppressions Ivan Skytte Jørgensen 2016-01-12 13:49:49 +01:00
  • 7e7f1a1074 Remove unused variable Ai Lin Chia 2016-01-12 10:47:06 +01:00
  • ae86f6b207 Remove unreachable code Ai Lin Chia 2016-01-11 23:21:51 +01:00
  • b4b69b83df Remove unused methods Ai Lin Chia 2016-01-11 23:12:54 +01:00
  • 924cad7f74 Remove code for enabling randomized user agent Ai Lin Chia 2016-01-11 17:15:31 +01:00
  • e79e59c875 Rename unused variables Ai Lin Chia 2016-01-11 17:09:06 +01:00
  • 2c14f659e4 Remove similar/unused Words::set methods Ai Lin Chia 2016-01-11 15:46:09 +01:00
  • 25af565df2 valgrind: size_... members were not all initialized Ivan Skytte Jørgensen 2016-01-12 11:46:17 +01:00
  • 75fd78f54c valgrind: avoid testing on unassigned member variables Ivan Skytte Jørgensen 2016-01-12 10:57:10 +01:00
  • 82e5ec7279 valgrind: avoid logging testing on unassigned member variables Ivan Skytte Jørgensen 2016-01-12 10:46:45 +01:00
  • 94c24d1cca valgrind: avoid logging testing on unassigned member variables Ivan Skytte Jørgensen 2016-01-12 10:39:01 +01:00
  • d76dad955d Test on ...valid before the real field Ivan Skytte Jørgensen 2016-01-11 17:55:59 +01:00
  • f528be1e31 Fix getRdbFromId() os the table is initialized correctly, and isn't a global variable Ivan Skytte Jørgensen 2016-01-11 16:41:25 +01:00
  • 5f2c213ae5 added output of termid to the term info page Brian Rasmusson 2016-01-11 15:58:55 +01:00
  • 273d681009 Removed unused stuff from posdb, like sorting entries for all numbers and facets Brian Rasmusson 2016-01-09 10:39:19 +01:00
  • d4a1d72cc0 Don't try to match implicit non-required phrases when verifying doc has query terms. Zak Betz 2016-01-08 10:09:34 -07:00
  • 2689ebc572 Remove emoticons from summary. Added system test to check for symbols removal. Fix bug in emoticon detection. Add unit test for emoticon detection. Ai Lin Chia 2016-01-08 15:20:42 +01:00
  • 9e100fbd13 Remove commented out code Ai Lin Chia 2016-01-08 10:31:05 +01:00
  • 8f35eaf4ac Removed tests for facebookdb and accessdb in Collectiondb::getCollnum() Ivan Skytte Jørgensen 2016-01-08 15:03:13 +01:00
  • 908f1204e0 Changed Msg5::m_waitingForList from char to bool Ivan Skytte Jørgensen 2016-01-08 14:56:09 +01:00
  • 44b8d6b0ad Removed write-only member Msg3::m_tfns and unused method Msg3::getTfn() Ivan Skytte Jørgensen 2016-01-08 14:46:34 +01:00
  • b8555d816d isPingServer() -> no-op Ivan Skytte Jørgensen 2016-01-08 14:38:24 +01:00
  • 4263920833 Fix query "the" and "the" not matching all of the terms. Zak Betz 2016-01-07 15:30:45 -07:00
  • fd2ddc8c97 Fix core dump when titleBuf is NULL Ai Lin Chia 2016-01-08 10:27:06 +01:00
  • 1428fed6e0 added a few more entries to our meta-tag whitelist Brian Rasmusson 2016-01-07 16:28:55 +01:00
  • c8edd053bd Removed unused Msg5::mi, and changed the void* piggybacked 'parent' to a proper typed pointer Ivan Skytte Jørgensen 2016-01-07 15:17:28 +01:00
  • b1b4e7c9a8 Sanity check in dumpTitledb() to avoid coredump deep inside Msg3::readList() Ivan Skytte Jørgensen 2016-01-07 15:02:43 +01:00
  • bc766c5ce1 Ignore pytest error Ai Lin Chia 2016-01-07 14:59:49 +01:00
  • f82e7bd44e Check in dumpTitledb() that the collection is known Ivan Skytte Jørgensen 2016-01-07 14:45:03 +01:00
  • 14e952076c Merge branch 'master' of https://github.com/privacore/open-source-search-engine Brian Rasmusson 2016-01-07 14:41:24 +01:00
  • afe57ab085 Removed unneeded data from posdb and a bit code cleanup Brian Rasmusson 2016-01-07 14:41:17 +01:00
  • 7d0fa2385d Don't get summary text from 'script' / 'style' tags Ai Lin Chia 2016-01-07 11:50:56 +01:00
  • 61e1c92ba8 Remove write only m_dist variable Ai Lin Chia 2016-01-07 11:35:54 +01:00
  • f4b024f558 Removing unreachable code Ai Lin Chia 2016-01-07 00:01:06 +01:00
  • c9dd535a93 Remove useless methods (no logic in it). Remove some set but not used variables. Remove unused parameter from functions. Ai Lin Chia 2016-01-06 17:25:42 +01:00
  • eea7ec4a3c Fix compilation warning (‘memset’ used with constant zero length parameter; this could be due to transposed parameters) Ai Lin Chia 2016-01-06 14:01:51 +01:00
  • f2f90ed257 Removed commented code Ai Lin Chia 2016-01-06 12:41:40 +01:00
  • ce9bd4a88f Make sure meta tags used for summary is not exactly the same as title Ai Lin Chia 2016-01-06 11:52:23 +01:00
  • 9b16d67cca ifdef'd posdb0001 delete checks Brian Rasmusson 2016-01-06 11:27:25 +01:00
  • 72a1628e6b More removal of isgoogle stuff Ai Lin Chia 2016-01-05 15:00:44 +01:00
  • 452b19a920 Fix documentation for RdbTree Ivan Skytte Jørgensen 2016-01-05 14:43:22 +01:00
  • b6caea4149 Merge branch 'master' of github.com:privacore/open-source-search-engine Ivan Skytte Jørgensen 2016-01-05 14:07:21 +01:00
  • 2136431365 use memset() instead of hand-rolled loop in RdbMem::init() Ivan Skytte Jørgensen 2016-01-05 14:07:12 +01:00
  • 9e497c522b Remove commented out codes. Remove 'static' variable (always false). Ai Lin Chia 2016-01-05 13:30:41 +01:00
  • 54f945952b Remove unused swoggleTable(s) methods Ai Lin Chia 2016-01-05 12:21:35 +01:00
  • cd095e66bc Remove FORMAT_PROCOG & related codes. Remove more scraping code/google detection code Ai Lin Chia 2016-01-05 12:17:17 +01:00
  • 517cc8d0d5 Remove monitor.cpp Ai Lin Chia 2016-01-05 11:15:43 +01:00
  • 105c69ebba Remove commented out codes Ai Lin Chia 2016-01-05 10:41:16 +01:00
  • a6b9495b2c Remove writeonly m_sectionPtrsEnd & unused method Ai Lin Chia 2016-01-04 18:42:44 +01:00
  • 6d1969d065 Revert some short->int16_t replacements Ivan Skytte Jørgensen 2016-01-05 11:39:00 +01:00
  • f6576f98aa Removed unused/uncommented RdbTree::oldLoad() and RdbTree::oldLoadBlock() Ivan Skytte Jørgensen 2016-01-04 16:07:46 +01:00
  • d4a90306a3 Use tag enum instead of hardcoded values (there are probably more hanging around in the code) Ai Lin Chia 2016-01-04 15:14:33 +01:00
  • af3fdb7ba1 Remove read only variable m_detectSubPhrases (it's always false) Ai Lin Chia 2016-01-04 14:13:23 +01:00
  • 02f1ac1ce6 Fix compilation warning Ai Lin Chia 2016-01-04 14:04:19 +01:00
  • 3cc7a032e5 Remove unused/'constant' variable in Matches::addMatches. Modify int16_tcuts & similar to shortcuts & similar. Ai Lin Chia 2016-01-04 13:55:46 +01:00
  • db69eaf44a Removed unused members of SpiderRequest Ivan Skytte Jørgensen 2016-01-04 15:01:41 +01:00
  • a653ca502b Changed 1-bit fields in SpiderRequest and SpiderReply to unsigned Ivan Skytte Jørgensen 2016-01-04 14:12:06 +01:00
  • c1a02194bc Make sure when we reuse TcpSockets, the correct hostname is set as well Ai Lin Chia 2016-01-04 11:14:47 +01:00
  • efdaa5aa4f Removed unused member XmlDoc::m_seoDebug Ivan Skytte Jørgensen 2015-12-29 16:24:42 +01:00
  • 725df6b6e5 Removed unused member Msg39::m_seoDebug Ivan Skytte Jørgensen 2015-12-29 16:17:44 +01:00
  • 23161bdc81 Documented hash-group 10 Ivan Skytte Jørgensen 2015-12-29 14:39:44 +01:00
  • d2088e3cde Removed unused Posdb::makeKey48() Ivan Skytte Jørgensen 2015-12-28 15:49:23 +01:00
  • 15af693a98 Added 'term info' entry in search results Brian Rasmusson 2015-12-28 14:55:22 +01:00
  • 360d754fe8 Added cleandb target Ivan Skytte Jørgensen 2015-12-28 13:19:30 +01:00
  • ba0e355a7a Now creates a new log file when reaching 1 gigabyte, not only when running as daemon. Renamed log backup filename to logXXX-bakYYYYMMDD-HHMISS Brian Rasmusson 2015-12-28 12:20:49 +01:00
  • 0948c237ac Send hostname for SSL connection to cater for TLS SNI Ai Lin Chia 2015-12-22 15:46:01 +01:00
  • 971290aaad Ignore hidden file (.git directory) Ai Lin Chia 2015-12-22 13:40:04 +01:00
  • f64e1e0d7b Short circuit summary generation to use description/summary meta tag if present for "root" url Ai Lin Chia 2015-12-22 11:23:27 +01:00
  • c6840599bc Add 'dirty' when working directory is not clean Ai Lin Chia 2015-12-22 11:22:19 +01:00
  • 316c80174b Add clang format file (initial edition) Ai Lin Chia 2015-12-22 10:30:57 +01:00
  • b95504b6b2 Remove commented out carvetest Ai Lin Chia 2015-12-22 10:26:07 +01:00
  • e6b3544ca7 Remove unused Url::isExtensionIndexable Ai Lin Chia 2015-12-21 14:39:45 +01:00
  • fb94829497 Remove unused TcpServer::sendMsg method Ai Lin Chia 2015-12-21 14:30:02 +01:00
  • 3dd34f5459 Move LanguageIdentifier::guessCountryTLD to a static method. Remove constructor & destructor & global variable (g_langid). Ai Lin Chia 2015-12-21 11:03:28 +01:00
  • de89c2e98b Remove unused HTTPServer::getDoc method Ai Lin Chia 2015-12-21 10:54:25 +01:00
  • 5a697bee7b Remove BGCOLOR define Ai Lin Chia 2015-12-21 10:40:28 +01:00
  • 9172add438 Remove unused method in Address Ai Lin Chia 2015-12-21 10:11:54 +01:00
  • b61b0b55a6 Excessive logging ifdef'ed again Brian Rasmusson 2015-12-21 17:59:00 +01:00
  • 77e179e594 Rename unused XmlDoc members to ...unused.. Ivan Skytte Jørgensen 2015-12-21 15:18:22 +01:00
  • 526a4d77fe Removed remaining arc/warc parsing from XmlDoc Ivan Skytte Jørgensen 2015-12-21 15:01:23 +01:00
  • 38e207c289 Removed Url::isWarc() and Url::isArc() which are now unused Ivan Skytte Jørgensen 2015-12-21 15:00:32 +01:00
  • 8827d0246f Cleaned up more from arc/warc support Ivan Skytte Jørgensen 2015-12-21 14:51:12 +01:00
  • 9d7ff08bae gbmemcpy() and memcpy_ass() should actually call memmove() Ivan Skytte Jørgensen 2015-12-21 13:19:57 +01:00
  • db60a699f8 Added trace option for network messages and more debug log.. Brian Rasmusson 2015-12-20 20:34:00 +01:00
  • feb701de80 Store a list of current files before letting gb run. May ease debugging of file problems Brian Rasmusson 2015-12-18 23:32:49 +01:00
  • 8835d275c7 Added and corrected trace logging Brian Rasmusson 2015-12-18 21:30:20 +01:00
  • 4c7370c16b Make sure we abort when BigFile write fails Ai Lin Chia 2015-12-18 15:19:24 +01:00
  • e3819fbb11 Remove commented out codes Ai Lin Chia 2015-12-18 14:26:03 +01:00
  • 8760c667ec Remove old conversion program that doesn't compile anymore Ai Lin Chia 2015-12-18 14:05:46 +01:00
  • e9bbc86f17 Remove unused header. Remove codeblocks layout file. Ai Lin Chia 2015-12-18 13:37:02 +01:00
  • 3dcb75b126 Remove unused Syncdb Ai Lin Chia 2015-12-18 13:36:23 +01:00
  • eead00d77a use process.shutdownAbort to bail out on error Brian Rasmusson 2015-12-18 14:59:05 +01:00
  • 122769a476 Added logging and hard coded sanity checks to File.cpp Brian Rasmusson 2015-12-18 14:45:34 +01:00
  • f4b8598d73 Added some logging to RdbMap.cpp Brian Rasmusson 2015-12-18 14:17:03 +01:00
  • 64c9733b2a Minor fix in logging name of too-short file. Apparently File::getFilename() returns full-path for most files Ivan Skytte Jørgensen 2015-12-18 13:51:50 +01:00
  • 0155620de8 use of new shutdownAbort Brian Rasmusson 2015-12-18 12:10:27 +01:00