Forks/open-source-search-engine
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Commit Graph

  • ed6ed14196 fix rocket.jpg link Matt Wells 2014-08-27 18:29:51 -07:00
  • 9be5027fb0 expose thread pool sizes again for spider/query time tasks. mwells 2014-08-27 17:27:00 -07:00
  • d5ef8a36e7 fix crawldelay bug. we were ignoring it. mwells 2014-08-27 17:19:13 -07:00
  • 76fffc3a81 Merge branch 'testing' of github.com:gigablast/open-source-search-engine into testing mwells 2014-08-27 16:46:41 -07:00
  • 234b6ff831 use scp not rcp for 'gb installgb' etc so it works on redhat. mwells 2014-08-27 16:46:14 -07:00
  • 839509e406 fix performance graph. Matt Wells 2014-08-27 15:28:30 -07:00
  • 81006e4f74 updated dotgdbinit Matt Wells 2014-08-27 15:04:11 -07:00
  • ec4d4a95ef added dotgdbinit file for gdb Matt Wells 2014-08-27 15:03:30 -07:00
  • bdaf321c4c show process IDs on the stats page now. use scp not rcp for 'gb installgb' because it doesn't work on default redhat systems. Matt Wells 2014-08-27 14:10:50 -07:00
  • 8772e7fffe overhauled the main loop. (BIGLOOP) in Loop.cpp. sigtimedwait() was cutting it, it was queueing up too many DUPLICATE signals and overflowing the rt signal queue. now gb has its own real-time signal queueing logic that just sets the bit of the FDs that need attention. i think threaded reads/writes are better now too but the performance graph is broken so i need to fix that first. the threads page looks good though. overhaul this hopefully is a massive and stable performance improvement. Matt Wells 2014-08-27 14:07:13 -07:00
  • 317af88770 take out debug logs. mwells 2014-08-27 10:52:44 -07:00
  • f73195870b hacked up to debug why we're not getting signals on redhat etc. mwells 2014-08-27 10:37:03 -07:00
  • c79c2a59c4 updates mwells 2014-08-26 11:34:19 -07:00
  • bdc72c9e8a doc admin.html updates mwells 2014-08-26 08:50:27 -07:00
  • 042ec4b5cd show gigabits in xml/json feeds. update optimizing section in admin.html by adding a 'disable gigabits' section for making queries faster. mwells 2014-08-26 08:46:59 -07:00
  • d0ccbdd455 gui and doc updates mwells 2014-08-26 00:05:01 -07:00
  • 5c69d49176 fix html.html bug mwells 2014-08-25 21:49:32 -07:00
  • 314c678538 html cleanups mwells 2014-08-25 21:38:52 -07:00
  • b080a96301 fix widths of admin tool boxes mwells 2014-08-25 21:02:50 -07:00
  • 8e6d2db194 put basic/advanced tabs on top. mwells 2014-08-25 19:58:26 -07:00
  • 0c43fc82ea Merge branch 'diffbot-matt' into diffbot-testing Matt Wells 2014-08-25 17:16:31 -07:00
  • c3699f0da5 fix bugs found from qa tests. mwells 2014-08-25 14:34:30 -07:00
  • 6607cc2cbe added gbfieldmatch: operator for exactly matching full field names. case sensitive. uses gbfacetstr: values that were hashed at index time. example: gbfieldmatch:object.field:"Some Value" See help.html for more examples. mwells 2014-08-25 13:57:55 -07:00
  • 2800ce0e04 fix a few bugs pertaining to tags.uri:"" fix a while back. mwells 2014-08-25 12:40:51 -07:00
  • 15421908be Merge branch 'master' into testing mwells 2014-08-25 10:51:37 -07:00
  • 4dcc0ef369 fix the core better from getSpiderReplyMetaList2() Matt Wells 2014-08-25 07:06:25 -07:00
  • 1f9c230290 another bug fix for getSpiderReplyMetaList2() coring. mwells 2014-08-25 07:15:19 -07:00
  • 80f73bf297 Merge branch 'master' into testing Matt Wells 2014-08-23 07:30:17 -07:00
  • 425b2bb81b try to fix core dump that happens while spidering. mwells 2014-08-23 07:35:04 -07:00
  • 5d3fd80063 make it so we can dump tagdb to a wget-table list of urls to re-add tags to another tagdb. Matt Wells 2014-08-23 07:29:40 -07:00
  • bb7c6c29ce added image mwells 2014-08-22 03:53:57 -07:00
  • 42e5d02f31 Merge branch 'diffbot-testing' of github.com:gigablast/open-source-search-engine into diffbot-testing Matt Wells 2014-08-18 16:24:31 -07:00
  • 7befaac371 do not allow injects until clock is synced with host #0. Matt Wells 2014-08-18 16:23:57 -07:00
  • 346e21aa0a fix site: query operator and others from fixing the tags.uri: bug. Matt Wells 2014-08-18 10:25:55 -07:00
  • e45c0d32f6 Merge branch 'diffbot-testing' into testing mwells 2014-08-15 17:05:22 -07:00
  • 1d9411152f fix so tags.uri:foo/bar/baz is treated like tags.uri:"foo/bar/baz". also fixed another quote-related bug. Matt Wells 2014-08-15 14:49:21 -07:00
  • a62c971223 fix tags.uri:org/resource/Foo" query Matt Wells 2014-08-15 13:38:14 -07:00
  • 2af299da2c various fixes. prioritize process only urls over crawl urls to get data faster. do not merge on high negative rec concentration. we need to fix that more. allow simplified redirs again for custom crawls to avoid too many dups. raise crawlinfo delay from 1 sec to 5 secs to reduce network usage for now. add back in injection enabled parm, but hidden. Matt Wells 2014-08-15 10:27:50 -07:00
  • 786d73f481 help page updates mwells 2014-08-08 08:04:41 -07:00
  • 5b99c95c76 awesome update to the help page mwells 2014-08-08 07:48:11 -07:00
  • 50cc24e2be updating help table. still more work on query.cpp to do mwells 2014-08-07 22:02:55 -07:00
  • fb0c8f0c4d fix redirect detection error mwells 2014-08-07 17:11:58 -07:00
  • 31f8c13369 facet tests and fixes mwells 2014-08-07 16:35:36 -07:00
  • ffa64ea540 fix gbequalfloat:1.23 term mwells 2014-08-07 15:33:37 -07:00
  • 9b94ce2e40 Merge branch 'diffbot-testing' into testing mwells 2014-08-07 15:15:08 -07:00
  • 68d0f40723 always count seeds towards page crawl count even if does not match crawl pattern. mwells 2014-08-07 14:33:13 -07:00
  • 7a02fb3676 minor fix mwells 2014-08-07 11:43:01 -07:00
  • 734973bb81 do not increment pagedownloadsuccesses if url does not match crawl pattern. mwells 2014-08-07 11:22:31 -07:00
  • 5149f855af more nyt.com bug fixes mwells 2014-08-07 10:26:30 -07:00
  • 4c81ddedf3 add showbanned menu mwells 2014-08-07 08:08:20 -07:00
  • ab46517e13 two rows of filters mwells 2014-08-07 08:03:19 -07:00
  • ff14341f53 javascript updates for search filter bar mwells 2014-08-07 07:38:00 -07:00
  • 484cf91a60 more search filter bar updates mwells 2014-08-06 22:05:30 -07:00
  • 55e27f4fa5 search filter bar updates mwells 2014-08-06 20:43:48 -07:00
  • 177dbeb23d Merge branch 'testing' of github.com:gigablast/open-source-search-engine into testing mwells 2014-08-06 16:00:50 -07:00
  • 6a28250e94 get qa test working after nyt bug fix mwells 2014-08-06 16:00:25 -07:00
  • 470c487be4 get search filters actually working mwells 2014-08-06 08:12:05 -07:00
  • f59b05eec1 gui updates mwells 2014-08-05 21:21:04 -07:00
  • 947be58f10 Merge branch 'diffbot-testing' into testing mwells 2014-08-05 17:19:53 -07:00
  • cc1ceaaac2 fix nyt.com cookie redir bug. fixed bug when POSTing injection request with multipart/form-data. mwells 2014-08-05 17:04:11 -07:00
  • 90c7d4328b search filter bar updates mwells 2014-08-05 08:07:07 -07:00
  • 2c7e14d4ca go button updates mwells 2014-08-04 20:27:51 -07:00
  • 5bedf74e55 some more gui stuff mwells 2014-08-04 14:08:44 -07:00
  • 6e167b94b2 gui updates mwells 2014-08-04 07:17:13 -07:00
  • 90139509a6 gui updates mwells 2014-08-03 19:38:50 -07:00
  • 93152633db gui updates. fixed gigabits mwells 2014-08-03 17:25:22 -07:00
  • 0da51d595b gui fixes mwells 2014-08-03 13:19:32 -07:00
  • 13743acd5a gui updates mwells 2014-08-03 10:42:45 -07:00
  • 6eed25e27e added rocket.jpg mwells 2014-08-03 09:52:58 -07:00
  • d56b3d43a3 serp gui updates mwells 2014-08-03 09:52:41 -07:00
  • 429f50b3af great gui updates mwells 2014-08-02 22:19:15 -07:00
  • 6b7c7f4086 some gui updates mwells 2014-08-02 17:58:11 -07:00
  • 754cdf124b doc updates mwells 2014-08-02 11:57:33 -07:00
  • 2605462aed qa fixes mwells 2014-08-02 09:07:33 -07:00
  • 0200cce308 Merge branch 'master' into testing mwells 2014-08-01 21:08:41 -07:00
  • ba3e91b053 bring in core fix from testing mwells 2014-08-01 21:04:54 -07:00
  • 425ef7f425 fix core in getHeaderTagBuf() func mwells 2014-08-01 20:31:31 -07:00
  • ac39293091 qa updates. provide "gburl" for diffbot json objs so we can eject them with /admin/inject mwells 2014-08-01 14:29:41 -07:00
  • 1e47e32384 Merge branch 'diffbot-testing' into testing mwells 2014-08-01 14:06:25 -07:00
  • 2bd19968b2 Merge branch 'master' into testing mwells 2014-08-01 14:04:39 -07:00
  • 030e1c7c1f fix core when doing queries with &'s in them mwells 2014-08-01 14:04:05 -07:00
  • b2c734bacc fix some cores mwells 2014-08-01 12:10:33 -07:00
  • 146e45db56 try to fix some redirect issues mwells 2014-07-31 10:34:03 -07:00
  • de11166e1f fix core from running ./gb stop mwells 2014-07-31 07:15:34 -07:00
  • f64c7a0f0d facet fixes mwells 2014-07-31 07:04:20 -07:00
  • 257f225232 more facet fixes mwells 2014-07-30 17:00:33 -07:00
  • 037e60a9a0 fix facet gen hanging on socket bug mwells 2014-07-30 15:51:00 -07:00
  • c4174a0ca6 fix bug causing qa json facet test to fail mwells 2014-07-30 15:36:08 -07:00
  • 2d6fd06c27 fix facets over json docs. added json docs through add url and json queries to qa tests. mwells 2014-07-30 14:50:33 -07:00
  • 150277bc0f fix facets some more. added facet qa test. mwells 2014-07-30 12:17:53 -07:00
  • dff04eff45 fix facet/xpath lookup stuff. mwells 2014-07-30 10:41:21 -07:00
  • 546d726487 pretty ups mwells 2014-07-30 09:59:25 -07:00
  • 89249ecec8 gui updates mwells 2014-07-30 07:39:36 -07:00
  • 515110c087 gb -h updates. mwells 2014-07-30 06:46:38 -07:00
  • dc74386f87 pretty ups mwells 2014-07-29 21:29:01 -07:00
  • 3f584ecdaa query scrape fixes mwells 2014-07-29 19:55:31 -07:00
  • f405760a25 fix query scraping mwells 2014-07-29 19:51:41 -07:00
  • a9e61b5aca facet text lookup fixes. mwells 2014-07-29 19:32:27 -07:00
  • 312b39c059 lookup facet values to get their text representations. mwells 2014-07-29 16:17:18 -07:00
  • 4276700c8a fix qa tests mwells 2014-07-29 12:21:22 -07:00