Forks/privacore-open-source-search-engine
1
0
Fork 0
forked from Mirrors/privacore-open-source-search-engine
Code Pull Requests Activity

Commit Graph

  • a542e25a03 Split Msg40::gotSummary() into gotSummary and gotSummaries() Ivan Skytte Jørgensen 2018-01-11 17:58:12 +01:00
  • 73aa84d405 Removed the streaming feature ('stram' cgi parameter) Ivan Skytte Jørgensen 2018-01-11 17:45:53 +01:00
  • e00ef2469c Wordvariations: initial work on noun-prep-propernoun to propernoun-genetive noun rewrites Ivan Skytte Jørgensen 2018-01-11 15:21:12 +01:00
  • 3361bb25a3 Allow wordvariationsgenerators to log Ivan Skytte Jørgensen 2018-01-11 14:15:29 +01:00
  • 59cf9071e1 Help coverity/flexelint with an explicit gbshutdownLogicError() Ivan Skytte Jørgensen 2018-01-11 13:01:28 +01:00
  • 9e74c0031d Added implicit7non-obvious condition in if() Ivan Skytte Jørgensen 2018-01-11 12:51:29 +01:00
  • 4b1630444a Merge branch 'master' into dev-docprocess Ai Lin Chia 2018-01-11 10:51:50 +01:00
  • 55aeae8e3c Add more comment/logs Ai Lin Chia 2018-01-11 10:40:27 +01:00
  • 54e07dac0c Simplified PosdbTable::getMinTermPairScoreSlidingWindow() and made logic clearer Ivan Skytte Jørgensen 2018-01-09 13:41:51 +01:00
  • 5189129d0d Changed a few do-while loops into regular while loops Ivan Skytte Jørgensen 2018-01-09 12:49:07 +01:00
  • a54dce7b65 Make sure we don't hammer servers by having a firstIp check for docrebuild Ai Lin Chia 2018-01-08 17:39:58 +01:00
  • d09ad5027e Simplify code a bit in getTermPairScoreForAny() Ivan Skytte Jørgensen 2018-01-08 17:29:42 +01:00
  • 000858d318 Add new parameters for support of configurable delay & max pending docItem Ai Lin Chia 2018-01-08 16:32:56 +01:00
  • 0b7799524a Modify dump_redirect to dump first_ip as well Ai Lin Chia 2018-01-08 13:47:15 +01:00
  • 7a4fe44cf1 Fix unit test dev-cache Ai Lin Chia 2018-01-08 10:38:28 +01:00
  • 98ca4f1ded Initialize total_size, more logs for FxCache Ai Lin Chia 2018-01-08 10:38:02 +01:00
  • 8945ce26b3 Initialize array/buffer bestMinTermPairWindowPtrs in PosdbTable::getMinTermPairScoreSlidingWindow() Ivan Skytte Jørgensen 2018-01-05 17:14:00 +01:00
  • 26bc5ff50f Add max size check to FxCache & method to remove item from FxCache Ai Lin Chia 2018-01-05 17:01:41 +01:00
  • 2472217a7a Some helper functions Ai Lin Chia 2018-01-05 16:44:21 +01:00
  • 1ae2d6f669 Simplify loop-end-conditiion in PosdbTable::getMinTermPairScoreSlidingWindow() Ivan Skytte Jørgensen 2018-01-05 15:35:57 +01:00
  • 6aba535323 Merge branch 'master' into dev-cache Ai Lin Chia 2018-01-05 15:18:40 +01:00
  • aac2ba8a98 posdbtable: removed another m_bflags test Ivan Skytte Jørgensen 2018-01-05 14:57:34 +01:00
  • 34fa2b2581 Merge branch 'master' of github.com:privacore/open-source-search-engine Ivan Skytte Jørgensen 2018-01-05 14:55:21 +01:00
  • 5f956b64ef initialize QueryTerm::m_userNotRequired Ivan Skytte Jørgensen 2018-01-05 14:55:14 +01:00
  • 219bb34df9 posdbtable: removed a m_bflags test Ivan Skytte Jørgensen 2018-01-05 14:25:23 +01:00
  • f3dc9fca42 Merge branch 'master' into dev-rdblist Ai Lin Chia 2018-01-05 14:16:58 +01:00
  • a9fec07cb3 posdbtable: removed superfluous condition Ivan Skytte Jørgensen 2018-01-05 14:16:39 +01:00
  • 5cf404e413 XmlDoc::hashNeighborhoods(): goto-loop -> for-loop Ivan Skytte Jørgensen 2018-01-05 14:12:47 +01:00
  • 13fb07bde2 Comment that HASHGROUP_INLIST is apparently not used anymore Ivan Skytte Jørgensen 2018-01-05 14:06:29 +01:00
  • 3f57df637d Return qterm->m_userNotRequired in JSON output Ivan Skytte Jørgensen 2018-01-05 12:35:05 +01:00
  • 3ccf69790f Handle orange-printing of non-default float/double parameters correctly Ivan Skytte Jørgensen 2018-01-05 12:32:42 +01:00
  • 984c51448a Optimized compilation of InstanceInfoExchange Ivan Skytte Jørgensen 2018-01-05 12:30:01 +01:00
  • 68aec3a492 Add test for posdb getList while merging Ai Lin Chia 2018-01-05 11:56:21 +01:00
  • e0e24b6498 Fix merge list when posdb is merging Ai Lin Chia 2018-01-05 11:21:51 +01:00
  • 93247cee3c added [nrw] keyword so you can mark query words as not required. For testing purposes. Brian Rasmusson 2018-01-04 17:31:53 +01:00
  • 86679220c2 Better encapsulation of Posdbtable Ivan Skytte Jørgensen 2018-01-04 17:13:24 +01:00
  • 114e4d1b06 Removed unused PosdbTable::freeMem() Ivan Skytte Jørgensen 2018-01-04 17:06:13 +01:00
  • 2c83d3b89a Better encapsulation of Posdbtable Ivan Skytte Jørgensen 2018-01-04 17:05:31 +01:00
  • 27835033b8 Posdbtable: use gbmin()/gbmax() instead of if() Ivan Skytte Jørgensen 2018-01-04 16:50:21 +01:00
  • ecee0c04b1 Posdbtable: more std:.vector<> instead of plain arrays Ivan Skytte Jørgensen 2018-01-04 15:26:31 +01:00
  • 6efbe7d089 Posdbtable: more std:.vector<> instead of plain arrays Ivan Skytte Jørgensen 2018-01-04 15:20:45 +01:00
  • fc478d2041 Changed PosdbTable::m_bestMinTermPairWindowScore into a regular pass-in parameter Ivan Skytte Jørgensen 2018-01-04 14:58:24 +01:00
  • 5f04edd464 Changed PosdbTable:m_bestMinTermPairWindowPtrs from emmebr to normal pass-in-parameter Ivan Skytte Jørgensen 2018-01-04 14:48:54 +01:00
  • e33da80138 Dump more info in 'term' field in JSON output Ivan Skytte Jørgensen 2018-01-04 12:45:11 +01:00
  • 17ec77139c Make sure EDOCCONVERTFAILED is passed from child doc to main doc instead of remapping it to EINTERNALERROR Ai Lin Chia 2018-01-04 11:47:00 +01:00
  • bd43066178 Set g_errno to indexCode Ai Lin Chia 2018-01-03 11:00:14 +01:00
  • d80ebeb929 Prepare hopcount before using it Ai Lin Chia 2018-01-03 10:56:36 +01:00
  • 781ce6e988 Remove commented out code Ai Lin Chia 2018-01-03 10:56:20 +01:00
  • 8b1dda1328 Add EUDPTIMEDOUT to spider tmp error Ai Lin Chia 2018-01-03 10:55:56 +01:00
  • b63e80e07f Fix misleading comment Ivan Skytte Jørgensen 2018-01-02 17:11:31 +01:00
  • d2895145dc More logs Ai Lin Chia 2018-01-02 15:19:41 +01:00
  • 747995c6ca Make sure redirUrl is valid Ai Lin Chia 2018-01-02 15:03:41 +01:00
  • 1338f3e0b4 Fix redirurl Ai Lin Chia 2018-01-02 14:50:32 +01:00
  • 0235f2be34 Add ENOLINKTEXT_AREATAG to errname Ai Lin Chia 2018-01-02 14:30:15 +01:00
  • e606a4916f Only get current tag rec when we need to. Clear m_redirUrl when it's not a redirect Ai Lin Chia 2018-01-02 13:55:00 +01:00
  • 1c2df49a3e Make mlockall() call configurable Ivan Skytte Jørgensen 2018-01-02 12:23:24 +01:00
  • 7fd2817bbe Merge branch 'master' into dev-cache Ai Lin Chia 2017-12-29 16:15:38 +01:00
  • b7581f69e2 Merge branch 'master' of github.com:privacore/open-source-search-engine Ivan Skytte Jørgensen 2017-12-29 15:58:42 +01:00
  • dad12a1f4b Extended query-stop-words danish with interrogative relative pronouns and query-general-adverbs Ivan Skytte Jørgensen 2017-12-29 15:58:34 +01:00
  • f98ad9aa5b Merge branch 'dev-linkdb' Ai Lin Chia 2017-12-29 15:15:57 +01:00
  • eaffeef380 Split out danish query-stop-words Ivan Skytte Jørgensen 2017-12-29 15:04:15 +01:00
  • 4ef2e3050c Improved qstopword generation + dependencies Ivan Skytte Jørgensen 2017-12-29 14:56:10 +01:00
  • 544e1b12ae Do not treat unrecognized "something:something" queries as having fields. Ivan Skytte Jørgensen 2017-12-29 14:31:03 +01:00
  • c7946ad00b Use SiteGetter to get site instead of using host (wrong site hash is calculated when website have a non-default port) Ai Lin Chia 2017-12-29 14:13:42 +01:00
  • 11463f04c5 Log more info when debugging queries Ivan Skytte Jørgensen 2017-12-29 14:11:44 +01:00
  • 9be32fc9a2 Log more info when debugging queries Ivan Skytte Jørgensen 2017-12-29 14:05:06 +01:00
  • eae13e77de Removed unused parameter 'hasColon' from getFieldCode() Ivan Skytte Jørgensen 2017-12-29 12:58:00 +01:00
  • d4728fc24c We should use tagrec based on current url to get site tag instead of tagrec based on first url Ai Lin Chia 2017-12-28 16:42:18 +01:00
  • 2d29ac817c Merge branch 'master' into dev-linkdb Ai Lin Chia 2017-12-28 14:11:13 +01:00
  • 7bf2f51e04 Remove commented out codes Ai Lin Chia 2017-12-22 13:27:46 +01:00
  • 6e583b59a4 Add linkdb lookup page Ai Lin Chia 2017-12-28 14:10:06 +01:00
  • 4790ba8ff7 More log of initializing..initialized Ivan Skytte Jørgensen 2017-12-28 13:46:15 +01:00
  • 0134a4740b Fix more is8859-1 -> utf8 encoding errors in StopWords.cpp Ivan Skytte Jørgensen 2017-12-28 12:45:15 +01:00
  • 7983ec600b Remove commented out codes Ai Lin Chia 2017-12-22 13:27:46 +01:00
  • e112672b42 query-stop-words: use extern files if present, fallback to builtin defaults if not Ivan Skytte Jørgensen 2017-12-22 16:49:18 +01:00
  • 3292e55e7e NULL-terminate query-stop-word tables Ivan Skytte Jørgensen 2017-12-22 15:48:03 +01:00
  • 4c3a3551ae MOved query-stop-words arrays out from StopWords.cpp to textfiles and generate the static arrays at build time Ivan Skytte Jørgensen 2017-12-22 15:43:59 +01:00
  • 30c26c2d5b add word_variations to include dir Brian Rasmusson 2017-12-22 15:37:27 +01:00
  • 6d7ac22150 add sto and word_variations in libgb.a Brian Rasmusson 2017-12-22 15:37:02 +01:00
  • d205a6c09f Fix <ignored>-<normal>-<ignored> bigram queries Ivan Skytte Jørgensen 2017-12-22 14:45:49 +01:00
  • ee87ae36a7 Fix compilation error Ai Lin Chia 2017-12-22 14:37:49 +01:00
  • 88665396de Rename GbCache to FxCache Ai Lin Chia 2017-12-22 14:05:59 +01:00
  • e06c388e81 Remove commented out codes Ai Lin Chia 2017-12-22 13:27:46 +01:00
  • 8062aa9771 split dump linkdb option into two: dump linkdb by site or url Brian Rasmusson 2017-12-21 21:13:14 +01:00
  • 30c02bf166 added url hashes to info page Brian Rasmusson 2017-12-21 17:33:07 +01:00
  • 262312c350 Handle page-temp-min=0 properly (was going into inf/nan areas Ivan Skytte Jørgensen 2017-12-21 17:32:11 +01:00
  • bc341c7f9c DOn't output html in the middle of non-html (when eg.summaries timed out) Ivan Skytte Jørgensen 2017-12-21 14:01:19 +01:00
  • ea6727639f Changed PosdbTable::m_qiBuf from a SafeBuf into a normal std:.vector<> Ivan Skytte Jørgensen 2017-12-21 13:53:03 +01:00
  • 7b28994207 multicast should never send to twin if errcode is ENOLINKTEXT_AREATAG Brian Rasmusson 2017-12-21 10:15:48 +01:00
  • beb6489f6e output linkdb hostid and sitehash32 on info page Brian Rasmusson 2017-12-21 10:14:04 +01:00
  • 1fa42768d4 Merge branch 'master' into dev-language Ai Lin Chia 2017-12-19 11:27:08 +01:00
  • 2904584d04 Add const Ai Lin Chia 2017-12-19 11:25:45 +01:00
  • 8b3623a36d Merge branch 'master' of github.com:privacore/open-source-search-engine Ivan Skytte Jørgensen 2017-12-18 17:41:47 +01:00
  • 550350321c Initialize Query::m_bigramWeight/m_synonymWeight (make coverity happy) Ivan Skytte Jørgensen 2017-12-18 17:41:40 +01:00
  • 6c0fb6d81e Initialize WordVariationWeights::simple_spelling_variants Ivan Skytte Jørgensen 2017-12-18 17:36:34 +01:00
  • 65d26a1566 Merge branch 'master' of github.com:privacore/open-source-search-engine Ivan Skytte Jørgensen 2017-12-18 16:42:28 +01:00
  • b205aac14f bugfix: if last match in a document was the best then sometimes the document was eliminated Ivan Skytte Jørgensen 2017-12-18 16:39:31 +01:00
  • bbca41bd74 removed log line Brian Rasmusson 2017-12-18 16:33:26 +01:00
  • fbebd66a3c Improved site matching when getting site inlink info. SiteGetter adds a www prefix to domains without www. This domain is used when requesting link info through msg25. The check code would not be able to find links in pages not using the www prefix, although linkdb know there is a link. this resulted in the 'build: Got linknode=-1 < 0. Cached linker AAA does not have outlink to www.domain.com like linkdb says it should.' error. 'Fixed' by checking for both non-www and www versions of the link. Brian Rasmusson 2017-12-18 16:13:57 +01:00
  • 8324e5701b Log more progress during startup Ivan Skytte Jørgensen 2017-12-18 12:58:25 +01:00