Forks/privacore-open-source-search-engine
1
0
Fork 0
forked from Mirrors/privacore-open-source-search-engine
Code Pull Requests Activity

Commit Graph

  • dead88daff Fixed loggin Ai Lin Chia 2017-12-18 12:52:27 +01:00
  • 154cd5febf Add const Ai Lin Chia 2017-12-18 12:52:14 +01:00
  • 5d01a43a4d Merge branch 'master' into sto Ivan Skytte Jørgensen 2017-12-18 12:40:45 +01:00
  • 191e975655 Don't log ENOLINKTEXT_AREATAG as error in Multicast and don't try sending it to a twin. Added LogDebug option for Msg20. Brian Rasmusson 2017-12-16 16:00:24 +01:00
  • a90a6c536e output hash values for links in get_titlerec tool Brian Rasmusson 2017-12-16 11:57:36 +01:00
  • 6ed890237f Toned down error logs for Msg20 when they are really not errors. If inlink text cannot be fetched because the link is in an area tag, a new error code ENOLINKTEXT_AREATAG is used instead of EBADENGINEER. Added trace log for Msg25. Brian Rasmusson 2017-12-16 11:54:13 +01:00
  • 3065eded7f do not add www to url automatically when dumping linkdb Brian Rasmusson 2017-12-16 11:34:32 +01:00
  • 6415d78692 added more options to rank test page Brian Rasmusson 2017-12-15 16:17:36 +01:00
  • 896b6bc49e Merge branch 'master' into sto Ivan Skytte Jørgensen 2017-12-15 14:29:14 +01:00
  • 70a091f5cd Improved performance of STO loading (approx 30% faster now) Ivan Skytte Jørgensen 2017-12-15 14:28:51 +01:00
  • 1ac266f774 Rename GxLanguage to FxLanguage Ai Lin Chia 2017-12-15 12:56:29 +01:00
  • 095e51f04a Fix json response Ai Lin Chia 2017-12-15 11:23:25 +01:00
  • d1806623f0 fixed dump linkdb command. wrongly tried to interpret the url as a term id Brian Rasmusson 2017-12-15 11:08:04 +01:00
  • 941df60bcb Merge branch 'master' into sto Ivan Skytte Jørgensen 2017-12-14 17:29:40 +01:00
  • a747076eb5 Merge branch 'master' of github.com:privacore/open-source-search-engine Ivan Skytte Jørgensen 2017-12-14 17:23:11 +01:00
  • 753ed63b18 Handle overlong queries correctly. Ivan Skytte Jørgensen 2017-12-14 17:21:22 +01:00
  • bf61e50f8b Add JSON support for admin/tagdb page Ai Lin Chia 2017-12-14 16:11:36 +01:00
  • 55c3ff9df4 quelch debuglog about unused terms if the rdb list for the term was empty Ivan Skytte Jørgensen 2017-12-14 15:57:02 +01:00
  • c5714d0582 Merge branch 'master' of github.com:privacore/open-source-search-engine Ivan Skytte Jørgensen 2017-12-14 15:27:16 +01:00
  • 10fb99ded0 Calculate QueryWord::m_phraseLen correctly Ivan Skytte Jørgensen 2017-12-14 15:15:07 +01:00
  • 93e6051d9b only log 'checksum parsing inconsistency' as warning if comparing same titlerec versions, otherwise log as debug as what we hash may have changed between titlerec versions Brian Rasmusson 2017-12-14 14:06:25 +01:00
  • cf082f582b SafeBuf::jsonEncode() must check for ascii-0 Ivan Skytte Jørgensen 2017-12-14 14:03:20 +01:00
  • 01514550c7 Revert "Temp fix compilation error" Ivan Skytte Jørgensen 2017-12-14 13:56:03 +01:00
  • 66199ea28f Revert "Revert "Changed SafeBuf::jsonEncode() to no longer modify the source string for temporary NUL termination"" Ivan Skytte Jørgensen 2017-12-14 13:56:01 +01:00
  • 10a1a43ded We should not check redirUrl with canonical, only first url should be checked because we don't want to store canonical document under a different url Ai Lin Chia 2017-12-14 13:47:15 +01:00
  • ab62801d30 Create ssl key/cert before starting test Ai Lin Chia 2017-12-14 12:01:03 +01:00
  • 2323d88284 Temp fix compilation error Ai Lin Chia 2017-12-14 11:20:27 +01:00
  • adbc125483 Revert "Changed SafeBuf::jsonEncode() to no longer modify the source string for temporary NUL termination" Ai Lin Chia 2017-12-14 11:18:40 +01:00
  • a9950e58fd corrected logging of high port numbers Brian Rasmusson 2017-12-14 09:58:34 +01:00
  • 2d58a15135 made it configurable to check URL filters when handling queries Brian Rasmusson 2017-12-14 09:45:30 +01:00
  • cc09a0ff0b fixed url filter for dk Brian Rasmusson 2017-12-14 08:49:47 +01:00
  • 4f1ab01497 Add dump tool to dump redirected document Ai Lin Chia 2017-12-13 23:02:08 +01:00
  • ea14bd8162 fix qlang setting on new rank test page. -1 as 'any' confused gb Brian Rasmusson 2017-12-13 12:20:07 +01:00
  • 5ce9ec073a added safe-mode url filters for .dk domains only Brian Rasmusson 2017-12-13 10:42:40 +01:00
  • 139888385d adjusted selection values on new ranktest page Brian Rasmusson 2017-12-12 22:49:21 +01:00
  • c7f624dea5 added link to javascript cookie handling code Brian Rasmusson 2017-12-12 22:35:40 +01:00
  • b883156bd6 added page to easily test ranking parameters Brian Rasmusson 2017-12-12 22:29:27 +01:00
  • 171f8c874d http to https redirection on same host should be treated as simplified redirection Ai Lin Chia 2017-12-12 16:50:31 +01:00
  • 614aa72753 Permanent redirect should be treated as simplified redirect Ai Lin Chia 2017-12-12 16:49:52 +01:00
  • ab9ad2e196 Fix general page info page (get?page=1) Ai Lin Chia 2017-12-12 16:35:14 +01:00
  • 3a3b56fdbd Merge branch 'master' into sto Ivan Skytte Jørgensen 2017-12-12 16:46:13 +01:00
  • f98bcdb0b0 Fixed several cases where PosdbTable.cpp used uninitialized data Ivan Skytte Jørgensen 2017-12-12 16:19:15 +01:00
  • e6eef33f58 detect unused terms: ignore stopwords too Ivan Skytte Jørgensen 2017-12-12 15:41:28 +01:00
  • 3a793b24c8 Merge branch 'master' of github.com:privacore/open-source-search-engine Ivan Skytte Jørgensen 2017-12-12 15:21:55 +01:00
  • 0d48c04afb Detect when queryterms are unused by posdbtable and should have been used Ivan Skytte Jørgensen 2017-12-12 15:21:44 +01:00
  • ea7252f99a Check if parentUrl is there before using it Ai Lin Chia 2017-12-12 14:47:27 +01:00
  • 4eadf3e836 Merge branch 'master' of github.com:privacore/open-source-search-engine Ivan Skytte Jørgensen 2017-12-12 14:43:50 +01:00
  • 226d2ed8be Handle long/implicitly-truncated queries better Ivan Skytte Jørgensen 2017-12-12 14:43:12 +01:00
  • 98e1207aa0 More fixes for docrebuild Ai Lin Chia 2017-12-12 14:24:36 +01:00
  • bbf90d5b46 query: check m_userWeightForSynonym when generating synonyms Ivan Skytte Jørgensen 2017-12-12 14:10:50 +01:00
  • 618ec7d9ec Merge branch 'master' into sto Ivan Skytte Jørgensen 2017-12-12 14:06:50 +01:00
  • 18920967d0 Merge branch 'master' of github.com:privacore/open-source-search-engine Ivan Skytte Jørgensen 2017-12-12 14:06:36 +01:00
  • b3a4573860 query: test weights with almostEqualFloat(). Test wordWeight also when generating synonyms Ivan Skytte Jørgensen 2017-12-12 14:06:28 +01:00
  • eebe650fd1 Merge branch 'master' into sto Ivan Skytte Jørgensen 2017-12-11 17:07:33 +01:00
  • 76e6fe3320 Fixed merge from master Ivan Skytte Jørgensen 2017-12-11 16:55:02 +01:00
  • e632f1f420 Remove unused variable & commented out code Ai Lin Chia 2017-12-11 16:54:28 +01:00
  • 2707cf7862 Use LF_SAMESITE instead of LF_SAMEHOST when deciding to send tagrec or not (we could have different sites for the same host) Ai Lin Chia 2017-12-11 16:53:39 +01:00
  • e42305ec61 Merge branch 'master' into sqlite Ivan Skytte Jørgensen 2017-12-11 16:51:29 +01:00
  • aba03fe945 Removed unused QueryWord::m_level Ivan Skytte Jørgensen 2017-12-11 16:15:09 +01:00
  • 36b99cada4 Support [s] brackets for setting userweight on synonyms/word variations Ivan Skytte Jørgensen 2017-12-11 16:03:57 +01:00
  • 78425d243b Merge branch 'master' into sto Ivan Skytte Jørgensen 2017-12-11 15:41:57 +01:00
  • cf9ab92e9a user-weight as floating-point Ivan Skytte Jørgensen 2017-12-11 15:40:49 +01:00
  • adaa1fd81f Set qterm:m_termWeight correctly for original words and wikitionary-synonyms Ivan Skytte Jørgensen 2017-12-11 15:09:34 +01:00
  • 0302452f25 Merge branch 'master' into sto Ivan Skytte Jørgensen 2017-12-11 15:01:07 +01:00
  • 7a0db3a605 Merge branch 'sto' of github.com:privacore/open-source-search-engine into sto Ivan Skytte Jørgensen 2017-12-11 14:47:18 +01:00
  • 2358d250d9 gitignore: default_css.inc Ivan Skytte Jørgensen 2017-12-11 14:46:43 +01:00
  • 6ff1f91726 Moved rankign/scoring parameters into a single struct BaseScoringParameters Ivan Skytte Jørgensen 2017-12-11 14:44:58 +01:00
  • e800ea4b58 Merge branch 'master' of github.com:privacore/open-source-search-engine Ivan Skytte Jørgensen 2017-12-11 14:43:16 +01:00
  • 940559d375 Fix setting of SpiderRequest Ai Lin Chia 2017-12-11 14:08:16 +01:00
  • bd5fe9397c Remove useTimeAxis feature Ai Lin Chia 2017-12-11 12:25:12 +01:00
  • dd1e08f0cc Use BaseScoringParameters struct more places Ivan Skytte Jørgensen 2017-12-08 13:54:15 +01:00
  • 3b4d2758d3 Started moving scorign/ranking parameters to a single struct Ivan Skytte Jørgensen 2017-12-08 13:46:20 +01:00
  • 6e2fac79d5 Merge branch 'master' into sto Ivan Skytte Jørgensen 2017-12-08 12:56:16 +01:00
  • aa48008d11 Renamed members ..scoringWeights to derievedScoringWeights Ivan Skytte Jørgensen 2017-12-08 12:53:58 +01:00
  • 8878036e63 Renamed ScoringWeights to DerivedScoringWeights Ivan Skytte Jørgensen 2017-12-08 12:49:43 +01:00
  • 6f32f30527 Removed commented-out members of Msg39Request Ivan Skytte Jørgensen 2017-12-08 11:38:38 +01:00
  • 542bb564a6 Merge branch 'master' into sto Ivan Skytte Jørgensen 2017-12-08 11:31:47 +01:00
  • 143a1bdd5e Changed default for query expansion to off Ivan Skytte Jørgensen 2017-12-08 11:30:27 +01:00
  • 34a3ea8d1b More const members in SearchInput Ivan Skytte Jørgensen 2017-12-08 10:44:27 +01:00
  • 5ffe4072ec more const in PageResults (and removed a tmp-nul-termination) Ivan Skytte Jørgensen 2017-12-08 10:38:43 +01:00
  • 9889b3a307 Silence gcc warnign about class/struct forward decl. mismatch Ivan Skytte Jørgensen 2017-12-07 16:59:43 +01:00
  • f76f71f07b Slightly safer Msg39Reply::reset() Ivan Skytte Jørgensen 2017-12-07 16:53:48 +01:00
  • 5d5e333f89 Removed unused SearchInput::m_displayOutlinks Ivan Skytte Jørgensen 2017-12-07 16:30:18 +01:00
  • bd12a69f40 Merge branch 'master' into sto Ivan Skytte Jørgensen 2017-12-07 16:04:16 +01:00
  • 267daa0e96 Made siterankmultipler configurable Ivan Skytte Jørgensen 2017-12-07 15:34:11 +01:00
  • d6e6740143 Merge branch 'master' into sto Ivan Skytte Jørgensen 2017-12-07 13:23:46 +01:00
  • 331581ff0b gitignore: symlinks to .a files Ivan Skytte Jørgensen 2017-12-07 13:23:26 +01:00
  • 4af326bb07 Merge branch 'sto' of github.com:privacore/open-source-search-engine into sto Ivan Skytte Jørgensen 2017-12-06 16:46:07 +01:00
  • ee05b64999 word variations: look for other entries with same morphological unit id (kafeteater/cafétater) Ivan Skytte Jørgensen 2017-12-06 16:45:27 +01:00
  • 54111ae16c More fixes. Make sure all other related values are initialized for titlerec creation Ai Lin Chia 2017-12-06 13:45:45 +01:00
  • f4bcf46e85 More dump tools Ai Lin Chia 2017-12-06 12:11:21 +01:00
  • f834556aa2 Fix unittest Ai Lin Chia 2017-12-06 12:04:15 +01:00
  • 890964f1d8 Don't recalculate percentChangedPerDay if we're recycling content Ai Lin Chia 2017-12-06 12:01:22 +01:00
  • 6c8a9d2143 Make sure old terms are removed as well Ai Lin Chia 2017-12-06 11:06:45 +01:00
  • 10bdbb1af3 Merge branch 'master' into dev-robots Ai Lin Chia 2017-12-06 10:40:26 +01:00
  • 61be312b4e Make sure we rebuild with latest titlerec version Ai Lin Chia 2017-12-05 17:54:10 +01:00
  • 5e667a758f index prefix sitenoindex instead of site when we're not suppose to index the page Ai Lin Chia 2017-12-05 17:46:15 +01:00
  • c0f2276667 Fix typo in method name Ivan Skytte Jørgensen 2017-12-05 17:41:41 +01:00
  • 50f255ed60 Fix logging undefined value (m_termWeight) Ivan Skytte Jørgensen 2017-12-05 17:25:28 +01:00
  • 55105b2300 word variations: support simple spelling variants Ivan Skytte Jørgensen 2017-12-05 15:06:03 +01:00