Forks/open-source-search-engine
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Commit Graph

  • 4f7bd66315 fix core Matt 2014-11-27 14:33:04 -07:00
  • 8e315504a2 fix empty rdbcache bug of not enough buf mem. Matt Wells 2014-11-27 13:17:00 -08:00
  • 53207d80e8 add log warning if top tree too small Matt Wells 2014-11-27 08:52:03 -08:00
  • 23b8ebbc95 allocate enough nodes in top tree for more docid range splits. Matt Wells 2014-11-27 08:44:24 -08:00
  • 84bca1c2ca fix urgent password red box if we have a master ip. Matt 2014-11-27 08:20:15 -07:00
  • 477624be8a have to call g_loop.init() again as daemon process so we get sigalrms and quickpolls. Matt Wells 2014-11-27 07:08:13 -08:00
  • d3a25db329 take out swap out stuff Matt Wells 2014-11-27 06:31:20 -08:00
  • 1c3d87968b Merge branch 'diffbot-testing' into diffbot-matt Matt Wells 2014-11-27 06:29:26 -08:00
  • 0da7d2d2ff Merge branch 'diffbot-matt' of github.com:gigablast/open-source-search-engine into diffbot-matt Matt Wells 2014-11-27 06:26:08 -08:00
  • e2f6490232 fix diffbot reply deduping to exclude url: field in the json when doing deduping. Matt Wells 2014-11-27 06:25:27 -08:00
  • 6aa217ce1d quick fix Matt 2014-11-26 18:41:36 -07:00
  • cb33680b40 add make msg to install certain pkgs if make fails Matt 2014-11-26 18:41:05 -07:00
  • b4365b51f6 fix core from 64bit fixes Matt Wells 2014-11-26 17:27:39 -08:00
  • fa2ad505f2 'make gb32' compiles the 32-bit version. 'make' does 64-bit. Matt 2014-11-26 18:27:47 -07:00
  • bda364ab7b make compile on neo Matt Wells 2014-11-25 15:47:21 -08:00
  • 20dbfbb8ea 64 bit doc updates Matt 2014-11-25 16:25:43 -07:00
  • c111b18b29 a few hacks. temp hack for oom to split 4 ways for custom crawls Matt Wells 2014-11-25 15:01:23 -08:00
  • 5b92b5f6d5 now term freqs are almost exact for qatest123. sometimes an off by 1 bug. we should really call msg5 to get the list w/o thread and get a truly exact term freq for qatest123 for consistency. that would be in Posdb.cpp::getTermFreq() Matt 2014-11-25 15:54:15 -07:00
  • ea67c688b9 fixed a couple really nasty mem leak bugs from new facet code Matt 2014-11-25 11:00:27 -07:00
  • adcef39376 Merge branch 'diffbot-testing' into diffbot-matt Matt 2014-11-20 16:53:07 -08:00
  • 266d97608a fix a few more 64-bit conversion cores Matt 2014-11-20 16:12:18 -08:00
  • bf2013345d fix up diskpagecache. how did it work before without storing the vfd? because linked list was over many different vfds, but the map from one diskpage to a mem offset was specific to each vfd. Matt 2014-11-20 15:05:34 -08:00
  • 958e7db0c2 fix rdbcache bug from 64bit conversino i guess Matt 2014-11-19 10:31:31 -08:00
  • 83809c5e8a fix iconv mem free core. added 64-bit arch notice to output. Matt 2014-11-18 17:23:53 -08:00
  • c5989f4c4c fix new simplied Inlinks code some more Matt 2014-11-18 17:10:48 -08:00
  • 2977845375 simplify Inlinks class in LinkInfo.cpp. fix some more 64-bit related cores. Matt 2014-11-18 16:50:31 -08:00
  • 0032052930 doc spacing Matt 2014-11-18 13:28:48 -08:00
  • d9f129dcf7 fix "4" bug in RdbBase.cpp Matt 2014-11-18 11:58:53 -08:00
  • 9ec698a543 mostly doc updates Matt 2014-11-18 11:36:03 -08:00
  • 66e9abdfe5 fix gbsort core Matt 2014-11-17 18:32:44 -08:00
  • 8f8fb75139 this is about it for the text subs Matt 2014-11-17 18:28:23 -08:00
  • 4e8a42e024 text replacements for bad int32_t substitutions Matt 2014-11-17 18:24:38 -08:00
  • dbd8af0eaa -O4 put backs in makefile. efence off. Matt 2014-11-17 18:14:13 -08:00
  • 931a1c4bc6 good checkpoint. quite a few fixes. Matt 2014-11-17 18:13:36 -08:00
  • df35b61caa add libz64.a Matt 2014-11-17 14:53:59 -08:00
  • a1dec9b5d9 updated to a new libz64.a. updated zconf.h and zlib.h files and had to re-add g_quickPollFunc() into zlib's deflate.c Matt 2014-11-17 14:53:15 -08:00
  • 994dc02d62 make user weight a float, not int Matt 2014-11-17 14:46:31 -08:00
  • 5185afd8dd added 64 bit libiconv64.a Matt 2014-11-14 17:34:11 -08:00
  • 4a0554c76f more 64bit fixes Matt 2014-11-14 17:30:32 -08:00
  • 702785a8ee disable collection swapping temporarily for tagdb updates Matt 2014-11-13 15:36:10 -08:00
  • 87544e4895 fix rebuild bugs. Matt 2014-11-13 15:03:57 -08:00
  • 715dc8d916 fix rebuild code some more Matt 2014-11-13 13:55:29 -08:00
  • b529ed834e reduce log spam Matt 2014-11-13 13:14:39 -08:00
  • 69ef3c14ef fixes for repair/rebuild functionality. more to come. Matt 2014-11-13 13:04:28 -08:00
  • 8d5da5a2ba default query intersection threads to on for now Matt Wells 2014-11-12 20:47:40 -08:00
  • c6605d7b33 64 bit somewhat working at runtime. need to test all functionality to make sure. fixes are pretty trivial. Matt 2014-11-12 19:18:25 -08:00
  • 8a73cbe946 added a couple more facet examples Matt Wells 2014-11-12 17:22:21 -08:00
  • a68d9be856 cleanup all warning when not using -m32 Matt 2014-11-12 14:11:27 -08:00
  • 4c19453ea9 working with -m32 for basic testing. compiles for 64-bit. Matt 2014-11-12 11:38:37 -08:00
  • dd6761b95c fix bad sprintf Matt 2014-11-11 16:17:48 -08:00
  • ab58809811 no longer piggyback gbfieldmatch: on facet termlists because it needs to be super fast. Matt 2014-11-11 13:16:08 -08:00
  • 96b8197ad3 now it compiles with -m32 Matt 2014-11-10 14:45:11 -08:00
  • 444ed14cde reduce mem usage in rdbmap. useful for when there are thousands of tiny collections. Matt Wells 2014-11-07 08:49:08 -08:00
  • 4a91b5da9a fix core Matt Wells 2014-11-06 20:38:34 -08:00
  • 23c565afc8 fix a couple of cores. reduce memory usage significantly by not pre-allocating some per-collection hashtables. Matt Wells 2014-11-05 09:36:42 -08:00
  • 4eed706bd4 Merge pull request #32 from emmanuelcharon/diffbot-testing Gigablast 2014-11-04 19:26:53 -07:00
  • 790c525820 rename diffbotHopcount to maxHops emmanuelcharon 2014-11-04 16:05:20 -08:00
  • 7f8616f003 instructions update Matt Wells 2014-11-04 15:27:59 -08:00
  • bac9074f47 Merge branch 'diffbot-testing' into diffbot Matt Wells 2014-11-04 15:25:19 -08:00
  • 726e8a8316 Merge branch 'diffbot' into diffbot-testing Matt Wells 2014-11-04 15:20:32 -08:00
  • 118a5c1a33 fix gbfieldmatch:pageUrl:"http://...." query. problem with the : in the http:// and problem with Query.cpp converting pageUrl to lower case when it was hashed as "pageUrl" in XmlDoc::hashFacet2(). Matt Wells 2014-11-04 13:53:36 -08:00
  • ca77393372 fix core when top tree's setNumNodes tried to allocate too many nodes. Matt Wells 2014-11-03 18:47:37 -07:00
  • 205ef500bd Merge pull request #30 from Yeeler/develop Gigablast 2014-11-03 14:01:18 -07:00
  • 92bb93fbab Merge pull request #29 from emmanuelcharon/diffbot-testing Gigablast 2014-11-03 13:59:45 -07:00
  • 922bcc2133 fix core Matt Wells 2014-11-01 06:34:24 -07:00
  • 95f6dcf4f7 Merge branch 'diffbot' of github.com:gigablast/open-source-search-engine into diffbot Matt Wells 2014-11-01 06:18:20 -07:00
  • 45972d9837 disregard CONNECT requests for now Matt Wells 2014-11-01 06:17:36 -07:00
  • c29dedd714 added diffbotHopcount parameter for diffbot crawl and bulk jobs, also updated PageCrawlbot.cpp emmanuelcharon 2014-10-31 16:34:31 -07:00
  • dbd3898cf0 fix a couple cores Matt Wells 2014-10-31 13:36:07 -07:00
  • cc9dfc6e45 parser was not capturing negative sign so gbmin: and gbmax: and gbminint: etc. were not working for negative numbers. should work now. Matt Wells 2014-10-31 13:13:27 -07:00
  • 4a4a5e7f03 added Codeblocks project file Yeeler 2014-10-31 11:00:18 -07:00
  • a7dab95993 Merge branch 'diffbot-testing' of github.com:gigablast/open-source-search-engine into diffbot-testing Matt Wells 2014-10-31 07:31:19 -07:00
  • 722a79b384 Merge branch 'testing' into diffbot-testing Matt Wells 2014-10-31 07:31:06 -07:00
  • e7dd8f7956 replace long long with int64_t Matt Wells 2014-10-30 13:36:39 -06:00
  • b13f3d24d7 replaced unsigned long long with uint64_t Matt Wells 2014-10-30 13:30:39 -06:00
  • 17bcf512ae Merge branch 'testing' into diffbot-matt Matt Wells 2014-10-30 13:17:35 -06:00
  • fbb7e4fe44 fix coll passwords link Matt 2014-10-28 20:39:30 -06:00
  • a92ff328ae Merge branch 'diffbot-testing' of github.com:gigablast/open-source-search-engine into diffbot-testing Matt Wells 2014-10-28 18:25:32 -07:00
  • 254ad6895a to to fix OOM core Matt Wells 2014-10-28 18:25:16 -07:00
  • 9a12709e39 unfix Matt Wells 2014-10-27 20:31:06 -07:00
  • e458d4b9f7 Merge branch 'diffbot-testing' of github.com:gigablast/open-source-search-engine into diffbot-testing Matt Wells 2014-10-27 20:23:21 -07:00
  • 561f493e11 check for "urls" as well as "seeds" cgi parms before creating collection. Matt Wells 2014-10-27 19:59:59 -07:00
  • 5eb51b2d9e fix OOM condition a bit Matt 2014-10-27 16:17:37 -06:00
  • a8771f4e95 add 2nd collection passwords link mwells 2014-10-27 11:21:23 -06:00
  • d9e8f7e465 Merge pull request #26 from miketung/diffbot-testing Gigablast 2014-10-24 13:46:34 -06:00
  • 1dbfb55a8e User needs to specify seeds to create a crawlbot crawl. This fixes 2110. Mike Tung 2014-10-21 18:10:50 -07:00
  • 4d4faf3429 added query debug msgs for allocating TopNodes in TopTree. Matt Wells 2014-10-15 14:06:43 -07:00
  • 8d8570950b fix EXTREMELY slow unlinks and file renames on morpheus because of sync() cmd. Matt Wells 2014-10-14 11:01:38 -07:00
  • d933a3f038 fix &n=10000000 fix some more Matt Wells 2014-10-14 10:04:09 -07:00
  • 625c5e5790 fix squid proxy balancing code some more. fix death queries when msg2 had docs wanted of 0 because all termlists were empty. Matt Wells 2014-10-14 08:04:23 -07:00
  • 4914098e5c Merge branch 'diffbot-testing' of github.com:gigablast/open-source-search-engine into diffbot-testing Matt Wells 2014-10-13 20:51:42 -07:00
  • 393cd90c63 remove log debug msg Matt Wells 2014-10-13 20:51:22 -07:00
  • 1f9e38057c Merge pull request #24 from miketung/diffbot-testing Gigablast 2014-10-13 20:50:24 -06:00
  • f14552e194 Remove mobile user-agents to prevent fetching mobile version of page. Mike Tung 2014-10-13 19:36:34 -07:00
  • ef0e37d5a4 remove debug prints Matt Wells 2014-10-13 17:05:39 -07:00
  • bc4750b28c distribute diffbot proxy requests over all hosts. Matt Wells 2014-10-13 16:48:09 -07:00
  • 18d3b210ce added some debug logic. (will remove shortly) fix for mike's commit. Matt Wells 2014-10-13 16:36:43 -07:00
  • 843e96a759 Merge pull request #23 from miketung/diffbot-testing Gigablast 2014-10-13 16:53:52 -06:00
  • 9ad0f84d03 Enable facets for site Mike Tung 2014-10-13 15:41:49 -07:00
  • 162e89b2d5 return error if client tries to use https for squid proxy right now. Matt Wells 2014-10-10 07:47:23 -07:00