Forks/open-source-search-engine
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Commit Graph

  • 03afb81f10 Merge branch 'testing' into diffbot-testing mwells 2015-01-25 10:37:29 -07:00
  • ed36c0c073 fix gb memtest mwells 2015-01-25 10:36:43 -07:00
  • c3290393f9 show just backtrace in hex on core Matt 2015-01-25 08:46:27 -07:00
  • 430a96b2c8 followup fix Matt 2015-01-24 16:11:06 -07:00
  • 6d1341a533 fix cmd line injections for itemlist.txt support. Matt 2015-01-24 16:09:55 -07:00
  • 41ec645940 Merge branch 'testing' Matt 2015-01-24 10:57:55 -07:00
  • 04c7f7be42 fix misclicks banning sites Matt 2015-01-24 09:06:10 -07:00
  • b28f215afe pretty ups Matt 2015-01-23 23:03:10 -07:00
  • a2ce92cd71 fix core Matt Wells 2015-01-23 18:59:50 -08:00
  • 5a910ce2d9 Merge branch 'diffbot-testing' into diffbot Matt Wells 2015-01-23 17:33:51 -08:00
  • 81937dbec0 fix issue of not getting back all the docids when n is very high and it is limited to termlist sizes. Matt 2015-01-23 18:29:31 -07:00
  • 8ee72d69e2 Merge branch 'diffbot-testing' into diffbot Matt Wells 2015-01-23 07:31:18 -08:00
  • 9d355c3e9d Merge branch 'diffbot-testing' of github.com:gigablast/open-source-search-engine into diffbot-testing Matt Wells 2015-01-23 07:31:00 -08:00
  • 9c6912fa32 fix facet query that was breaking. fix click here to show all results link. Matt Wells 2015-01-23 07:30:43 -08:00
  • 83f8b74632 nothing Matt 2015-01-23 08:17:59 -07:00
  • 06f756eef2 try this Matt 2015-01-23 07:31:10 -07:00
  • 1a5461e83c profiler fixes Matt 2015-01-23 07:30:22 -07:00
  • b5f7441d3b fix profiler Matt 2015-01-23 07:27:17 -07:00
  • a762fe29e7 fix profiler some more Matt 2015-01-23 07:24:39 -07:00
  • edefae35d1 Merge branch 'diffbot-testing' into diffbot Matt 2015-01-22 20:12:36 -07:00
  • 5d403007b3 fix gb -r flag again to show 'x' in hosts table. Matt 2015-01-22 19:23:07 -07:00
  • a9bd0dd48a add help key for new U in hosts table. Matt 2015-01-22 15:58:36 -07:00
  • 72b6546ed9 fix some smoke tests Matt 2015-01-22 15:53:04 -07:00
  • faaaf3cb89 smoke test for query fix Matt 2015-01-22 14:56:51 -07:00
  • fe14079ffe show shards with excessive udp slots to detect jam up. Matt 2015-01-22 14:47:30 -07:00
  • ae220ce736 show addr if unknown in profiler Matt 2015-01-22 13:35:20 -07:00
  • 010468f69e some optimizations for speed Matt 2015-01-22 13:04:42 -07:00
  • c05efc2601 fix profiler paths to show full call stack Matt 2015-01-22 12:58:16 -07:00
  • aeaff79036 fix stack smash from 64-bit conversion some time ago Matt 2015-01-22 12:47:36 -07:00
  • b234a4c7c6 minor updates Matt 2015-01-22 11:38:22 -07:00
  • f931871a66 Merge branch 'testing' into diffbot-testing Matt 2015-01-22 11:35:48 -07:00
  • da86470143 fix query bug. Matt 2015-01-21 16:39:24 -07:00
  • 89d2908f5c fix gb -l -r Matt 2015-01-21 07:30:01 -07:00
  • 5218fc0dba fix if warc rec too big when injecting Matt 2015-01-20 23:48:56 -07:00
  • eb2a449379 Merge branch 'diffbot-testing' into testing Matt 2015-01-20 19:13:33 -07:00
  • bc6f065457 fix getFileSize(). fix warc injector. Matt 2015-01-20 19:12:58 -07:00
  • 980f1544a4 added quickpoll to help with 250MB diffbot reply. Matt Wells 2015-01-20 17:15:06 -08:00
  • db26c7ed76 more fixes for profiler. Matt 2015-01-20 17:02:10 -07:00
  • 44704a793d free profiler mem on exit Matt 2015-01-20 16:13:30 -07:00
  • 6e7b329cef speed up gb by fixing excessive calling to gettimeofday() system call. Matt 2015-01-20 16:06:01 -07:00
  • 52bf9f1ff0 integrate missed quickpolls into profiler so we can decrease overall latency. Matt 2015-01-20 15:38:42 -07:00
  • 48c9f34ea2 Merge branch 'diffbot-testing' of github.com:gigablast/open-source-search-engine into diffbot-testing Matt 2015-01-20 14:34:08 -07:00
  • a5c663670e built-in profiler now much better. works on 64bit too. uses addr2line. now we can speed things up easier. Matt 2015-01-20 14:33:32 -07:00
  • ecfcfb73c7 Merge branch 'testing' into diffbot-testing Matt 2015-01-20 12:47:51 -07:00
  • bb688bd50c max inject sockets upped Matt 2015-01-18 11:36:59 -07:00
  • 94a5814d60 default index spider status docs to off Matt 2015-01-18 10:40:05 -07:00
  • c0d1cb7df1 fix injects Matt 2015-01-18 10:33:01 -07:00
  • 88cb9e0010 update users Matt 2015-01-17 18:43:12 -07:00
  • 71809b4938 more updates Matt 2015-01-17 18:18:39 -07:00
  • d7b54727db remove double logo Matt 2015-01-17 15:17:15 -07:00
  • f6ab352356 up to 100MB Matt 2015-01-17 11:30:33 -07:00
  • d5b2ce2113 use warc timestamps for first index and spider times. Matt 2015-01-17 11:28:16 -07:00
  • 9d7a1a5868 ./gb inject <warcfile> <ip:port> now works somewhat. Matt 2015-01-17 11:17:58 -07:00
  • 8d7ddda03c Merge branch 'diffbot-testing' into testing Matt 2015-01-16 07:06:12 -08:00
  • 75fb004193 updates Matt 2015-01-15 20:58:56 -08:00
  • a010a64ec7 revenge of the germs update Matt 2015-01-15 20:42:27 -08:00
  • 87a78039e8 fix facets Matt Wells 2015-01-15 12:12:25 -08:00
  • 3df9a69f43 dump out facetval32 when doing a 'gb dump p' of a particular termid. Matt Wells 2015-01-15 11:07:21 -08:00
  • 51cda3bac0 fix malformed http reply header Matt Wells 2015-01-15 10:40:23 -08:00
  • e886f1bbac replace memcpy_ass with bcopy Matt 2015-01-14 14:12:55 -08:00
  • 988be47492 try to fix smoke test for testNotUpdatedContent in TestOnlyProcessIfNew class. Matt 2015-01-13 15:24:39 -08:00
  • 92cea033d0 nomeclature updates Matt 2015-01-13 13:38:31 -08:00
  • 0992a8cc5c do not show profiler link if not 32 bit arch Matt 2015-01-13 12:27:07 -08:00
  • c09886aef3 now real-time profiler is working again for 32-bit binaries because we use bcopy instead of memcpy so when profiler.cpp calls backtrace it can call its memcpy which is not async safe. Matt 2015-01-13 11:56:12 -08:00
  • 4969aa728e Merge branch 'diffbot-testing' of github.com:gigablast/open-source-search-engine into diffbot-testing mwells 2015-01-13 12:29:49 -07:00
  • 87285ba3cd use gbmemcpy not memcpy so we can get profiler working again since memcpy can't be interrupted and backtrace() called. mwells 2015-01-13 12:25:42 -07:00
  • cda39715f2 fix gigabit sample for json so we get some nice gigabits and fast facts. Matt 2015-01-12 18:06:42 -08:00
  • 18b5970e60 Merge branch 'diffbot-testing' of github.com:gigablast/open-source-search-engine into diffbot-testing Matt Wells 2015-01-12 14:00:05 -08:00
  • fb92064c64 just comments Matt Wells 2015-01-12 13:59:53 -08:00
  • 0ae882e1b5 fix query syntax help bugs related to facets Matt Wells 2015-01-12 10:41:02 -08:00
  • 19e493437a only print stack trace on core for 32-bit arches. won't work for 64bit right now. Matt 2015-01-10 13:03:12 -08:00
  • ef7b0c54fd log stack trace on core/segfault. Matt 2015-01-10 12:05:39 -08:00
  • f9ccc342a7 do not scan spiderdb for entries in waiting tree when spidering is turned off because it slows injections down. mwells 2015-01-10 09:19:14 -07:00
  • d541de8186 speed ups mwells 2015-01-09 14:56:32 -07:00
  • a935e68484 still do url extension check, but just remove certain ambiguous ones from the list to fix Mr.T. Matt 2015-01-08 11:09:16 -08:00
  • ec87655f93 no longer check for bad url extensions since that should now be done in url filters, either explicitly or by using the ismedia directive. Matt 2015-01-08 10:56:10 -08:00
  • c307cce330 update rebuild instructions Matt 2015-01-06 13:06:42 -08:00
  • 24fd6a1a26 fix log rotation logic. Matt Wells 2015-01-06 12:50:41 -08:00
  • b693fe1530 fix bugs related to restarting a cored shard during repair mode. need to be able to resume repair/rebuild scan. Matt Wells 2015-01-06 11:28:55 -08:00
  • 19c92339b3 fix core from corrupted root lang in title rec Matt Wells 2015-01-05 17:52:08 -08:00
  • e5b81cfb04 fix ping age being negative in hosts table bug. Matt Wells 2015-01-05 15:19:46 -08:00
  • f488be4ede make new logfile when current logfile hits 1GB. this will save disk space so we can delete the old log files that can be many GBs in size. Matt 2015-01-05 11:29:49 -08:00
  • c03ba31ec2 try to reduce log spam Matt 2015-01-05 11:03:49 -08:00
  • b7cb2b56e1 try to debug the core Matt Wells 2014-12-23 18:53:11 -07:00
  • e7dd98f54d fix print float pretty Matt Wells 2014-12-23 16:22:38 -08:00
  • 6e09035f46 try to fix performance slam when compiling like 640k facet list from each of 32 shards. hashtable was not hashing well and destroying the complexity. Matt Wells 2014-12-22 13:28:03 -07:00
  • 30ab1ec875 added a log statement to debug the ECTCPTIMEDOUT streaming core. make default qlang in searchinput.cpp parms.cpp be "" not NULL so it won't log qlang of "(null)" is NOT SUPPORTED. Matt Wells 2014-12-19 11:04:29 -08:00
  • b2bb5b4a45 Merge branch 'diffbot-testing' into testing Matt 2014-12-17 16:32:21 -08:00
  • e178c67f4b do not core on qa test fail Matt 2014-12-17 16:31:37 -08:00
  • 0f9cb96b91 speed up large query reindexes by using fake firstips limited to 0-64k to avoid excessive doledb winner generations. fix bug when injecting a content-less url that has the canonical tag in it. force it to go through. Matt Wells 2014-12-17 16:19:04 -08:00
  • d57f2264c4 more indicator fixes Matt Wells 2014-12-17 15:11:49 -08:00
  • f52e163fb0 fix a couple bugs. added out of sync indicator. Matt Wells 2014-12-17 14:28:32 -08:00
  • 465d30e0ee fix ping bug. Matt 2014-12-17 10:43:00 -08:00
  • ca68ae022a fix punct at beginning of term bug. Matt 2014-12-17 10:29:26 -08:00
  • 8c3f6a05c1 quick fix to prevent unnecessary re-INDEXING of diffbot replies. only reindex them when recycle diffbotreply is true AND doing query reindex with recycle set to true. Matt Wells 2014-12-17 06:45:43 -08:00
  • cad1c7c930 typo Matt Wells 2014-12-17 06:35:38 -08:00
  • 943beedf1b updated stats page to show # ooms, and took out # colls swapped out Matt Wells 2014-12-17 06:32:37 -08:00
  • 74ea82384f emergency core fix in XmlDoc::redoJSONObjects() from mismatched json count with title hashes for old doc. Matt Wells 2014-12-17 06:26:07 -08:00
  • 38c2671fd3 Merge branch 'diffbot' into testing Matt Wells 2014-12-16 19:22:35 -08:00
  • 2fd511f002 updates Matt Wells 2014-12-16 17:09:25 -08:00