Mirrors/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2025-06-22 04:16:09 -04:00
Code Issues Packages Projects Releases Wiki Activity

Commit Graph

  • a878c7982c prevent npe orbiter 2014-02-25 03:19:41 +01:00
  • e4eb87d924 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git orbiter 2014-02-25 02:16:37 +01:00
  • ced1a96f9c fixed error cache orbiter 2014-02-25 02:16:22 +01:00
  • 3ba81bd08a Merge origin/master reger 2014-02-25 00:24:10 +01:00
  • 4d896383db fix: use timeout = proxy.ClientTimeout in ProxyHandler (was 10sec fix) see http://bugs.yacy.net/view.php?id=236 reger 2014-02-25 00:23:06 +01:00
  • 072d4aa0c0 Updated German translation and Blacklist_p.html Michael Benz 2014-02-24 23:47:12 +01:00
  • 163cbceca5 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git orbiter 2014-02-24 23:43:03 +01:00
  • cfb647db6e - introduced a miss cache in ConcurrentUpdateSolrConnector - better usage of cache - bugfix for postprocessing orbiter 2014-02-24 23:42:50 +01:00
  • 2c8c51ce4b make use of new -config cmd-line parameter in reconfgureYACY.sh to asure pwd encoding is compatible with DIGEST auth. in future. reger 2014-02-24 23:19:17 +01:00
  • a87d8e4a8e changed caching of ConcurrentUpdateSolrConnector: it caches now also the url along with the load date. While this takes much more memory, it eliminates database lookups for getURL() requests, which happen equally often. This speeds up remote solr configurations. orbiter 2014-02-24 22:59:58 +01:00
  • f6e441dd77 refactoring orbiter 2014-02-24 21:01:56 +01:00
  • 76c53faeb2 removed unused code (HostStat) orbiter 2014-02-24 20:51:43 +01:00
  • d3a88eaecb introducing ConcurrentUpdateSolrServer for remote solr servers. Scaling of write buffers and update queue size is made according to assigned memory. orbiter 2014-02-24 20:26:02 +01:00
  • c3f6c06f2c removed host increment on stored documents from crawler (that was wrong) orbiter 2014-02-24 20:02:15 +01:00
  • f97428fe5d Merge branch 'master' of gitorious.org:yacy/icewindxs-rc1 Michael Peter Christen 2014-02-24 14:30:57 +01:00
  • 746aa32ad5 edit russian locale malykhin.dmitry 2014-02-24 16:50:28 +04:00
  • 809e976578 remove unused java imports form yacy.java reger 2014-02-24 05:19:40 +01:00
  • a9b06f8719 add a -config command line parameter e.g. -config "port=9090" "port.ssl=8043" - useful for remote installation to set any config file property - multipe parameter can be set at once, on Windows enclose parameter in doublequotes - special handling "adminAccount=adminuser:adminpwd" sets adminusername and md5 encoded admin-pwd reger 2014-02-24 05:16:31 +01:00
  • edc8e1c4de Finished translation of changed CrawlStartExpert_p.html Michael Benz 2014-02-24 04:38:28 +01:00
  • 0923b09216 fix: allow 4 character admin user name (was min 5 char) reger 2014-02-24 00:01:11 +01:00
  • 7253ca4607 Merge branch 'master' of gitorious.org:yacy/icewindxs-rc1 Michael Peter Christen 2014-02-22 09:51:02 +01:00
  • f8f0f6363d edit russian locale malykhin.dmitry 2014-02-22 07:42:05 +04:00
  • a86c2fe77d fixed usage of media flag when started by automated process Michael Peter Christen 2014-02-22 01:44:08 +01:00
  • 254a7ac66c fixed cleaning of index Michael Peter Christen 2014-02-22 01:35:01 +01:00
  • 28a7b42e6b removed warning "sun.misc.BASE64Encoder is internal proprietary API and may be removed in a future release" Michael Peter Christen 2014-02-22 00:52:49 +01:00
  • 046f5a03cb one more SolrIndexSearcher bugfix Michael Peter Christen 2014-02-21 23:48:56 +01:00
  • 78c01b3eff fix for 'AlreadyClosedException: this IndexReader is closed' sixcooler 2014-02-21 17:28:32 +01:00
  • f11314aae7 Improved German de.lng translation and fixed adresses -> addresses in \htroot\CrawlStartScanner_p.html Michael Benz 2014-02-20 08:35:51 +01:00
  • f0eec6d0f3 Merge branch 'master' of git://gitorious.org/~copro/yacy/copros-rc1 Michael Peter Christen 2014-02-20 00:50:48 +01:00
  • 1b5e3d523a better control over close-state of remote solr connections Michael Peter Christen 2014-02-20 00:39:19 +01:00
  • 6278af4993 Edit German de locale and improved translation Michael Benz 2014-02-20 00:32:20 +01:00
  • 1a364572a5 fix for "org.apache.solr.core.SolrCore Too many close [count:-1] on org.apache.solr.core.SolrCore@51af7c57" -error Michael Peter Christen 2014-02-20 00:03:35 +01:00
  • 69391e5d9e changed strategy to test existence of documents in Solr: using the update time. The reason for that is a better caching for the crawler double-check, which needs the update time for crawler steering. Michael Peter Christen 2014-02-19 04:03:45 +01:00
  • 790f103f32 delete fail-docs during postprocessing to prevent that they will appear again and stay in postprocessing forever. Michael Peter Christen 2014-02-18 01:38:56 +01:00
  • 745d6d1c64 Merge branch 'master' of ssh://gitorious.org/yacy/rc1 Michael Peter Christen 2014-02-17 23:59:48 +01:00
  • ec598991a4 edit russian locale malykhin.dmitry 2014-02-18 02:01:34 +04:00
  • a02e33dcb6 add edit-link to PK field of table admin reger 2014-02-16 02:26:11 +01:00
  • c69630c522 edit russian locale r 2014-02-15 07:29:49 +04:00
  • 66639cb703 Fixed my error Anatoliy Evladov 2014-02-14 14:12:25 +04:00
  • baaea6dedc Edit ru locales Anatoliy Evladov 2014-02-14 14:09:25 +04:00
  • ff656ce860 explicit call to optimize to add a expungeDeleted flag Michael Peter Christen 2014-02-12 01:01:23 +01:00
  • 9eb668e951 enhanced the resource observer The resource observer is now able to recognize free disk space AND available space for YaCy. The amount of space which is assigned for YaCy are defined in new settings in the configuration file. Furthermore, there is now a cleanup process which deletes files in case that an autodelete is activated. The autodelete is now BY DEFAULT ON if the disk space is low, which means that YaCy starts to delete documents when the disk is full! Michael Peter Christen 2014-02-12 01:00:44 +01:00
  • fbee98c06f fixed shortcut self-reference bug Michael Peter Christen 2014-02-11 22:14:46 +01:00
  • e7a29a2851 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git Michael Peter Christen 2014-02-11 22:03:46 +01:00
  • cb2c25d930 in case that the crawler is running and the search user is the peer admin, we expect that the user wants to check recently crawled document to ensure that recent crawl results are inside the search results, we do a soft commit here. Michael Peter Christen 2014-02-11 22:02:10 +01:00
  • bf97e38b83 removed clearURLIndex, which is a stub remaining from the old metadata database and not needed any more Michael Peter Christen 2014-02-11 22:01:25 +01:00
  • 14764632b5 clear solr caches in case that an exception occurrs. The reason behind this hack is the occurrence of Exceptions like: W 2014/02/11 18:51:33 ConcurrentLog GC overhead limit exceeded java.io.IOException: GC overhead limit exceeded at net.yacy.cora.federate.solr.connector.AbstractSolrConnector.getDocumentById(AbstractSolrConnector.java:334) at net.yacy.cora.federate.solr.connector.MirrorSolrConnector.getDocumentById(MirrorSolrConnector.java:173) at net.yacy.cora.federate.solr.connector.ConcurrentUpdateSolrConnector.getDocumentById(ConcurrentUpdateSolrConnector.java:415) at net.yacy.search.index.Fulltext.getMetadata(Fulltext.java:331) at net.yacy.search.index.Fulltext.getMetadata(Fulltext.java:317) at net.yacy.search.query.SearchEvent.pullOneRWI(SearchEvent.java:1024) at net.yacy.search.query.SearchEvent.pullOneFilteredFromRWI(SearchEvent.java:1047) at net.yacy.search.query.SearchEvent$3.run(SearchEvent.java:1263) Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded at java.util.Arrays.copyOfRange(Arrays.java:3077) at java.lang.StringCoding.decode(StringCoding.java:196) at java.lang.String.<init>(String.java:491) at java.lang.String.<init>(String.java:547) at org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.readField(CompressingStoredFieldsReader.java:187) at org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.visitDocument(CompressingStoredFieldsReader.java:351) at org.apache.lucene.index.SegmentReader.document(SegmentReader.java:276) at org.apache.lucene.index.BaseCompositeReader.document(BaseCompositeReader.java:110) at org.apache.lucene.index.IndexReader.document(IndexReader.java:436) at org.apache.solr.search.SolrIndexSearcher.doc(SolrIndexSearcher.java:657) at net.yacy.cora.federate.solr.connector.EmbeddedSolrConnector.SolrQueryResponse2SolrDocumentList(EmbeddedSolrConnector.java:230) at net.yacy.cora.federate.solr.connector.EmbeddedSolrConnector.getDocumentListByParams(EmbeddedSolrConnector.java:320) at net.yacy.cora.federate.solr.connector.AbstractSolrConnector.getDocumentById(AbstractSolrConnector.java:330) ... 7 more orbiter 2014-02-11 20:56:40 +01:00
  • bc28247089 Added methods in resource observer to calculate the available and the occupied disc space. These values are also shown on the status page. The disc space calculation shall be used for a disk-limitation of the search index. Michael Peter Christen 2014-02-11 03:20:03 +01:00
  • 0dda979801 adopted network image drawing to increased number of peers Michael Peter Christen 2014-02-11 00:53:10 +01:00
  • 365f77ea8c make internal page links relative to ease any future development for context aware servlets note also http://bugs.yacy.net/view.php?id=106 reger 2014-02-10 21:40:42 +01:00
  • 0e10cd18f7 upd: Maven pom to dev v1.69 reger 2014-02-10 21:25:01 +01:00
  • fbfee8c583 add forgotten solrj-4.6.1 sixcooler 2014-02-10 14:48:07 +01:00
  • 2dba7550f3 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git Michael Peter Christen 2014-02-10 12:59:54 +01:00
  • ca8b100f96 run the cleanup process even when load is high, do postprocessing even if load > 1 (but < 2) but only if there is enough memory (now: 0.5 GB RAM available). The memory amount of the postprocessing is the cause that systems block because they run into a frequent-GC chain which almost locks the peer. If running with enough memory, the postprocessing is fast and not damaging to the system. Because the required RAM of 0.5 GB is never available in default setting, the postprocessing will not run if the peer is not reconfigured to use more memory. Michael Peter Christen 2014-02-10 12:59:30 +01:00
  • 7eab0a2cb8 replaced guava-15.0 with guava-16.0.1 Michael Peter Christen 2014-02-10 11:55:40 +01:00
  • cb4fd27bbe replaced pdfbox 1.8.3 with 1.8.4 Michael Peter Christen 2014-02-10 11:49:48 +01:00
  • 4fcddcf13b replaced solr 4.6.0. with 4.6.1 Michael Peter Christen 2014-02-10 11:42:11 +01:00
  • ea63be019c next dev release Michael Peter Christen 2014-02-10 11:29:22 +01:00
  • dd7f7def7c upd: Maven pom reger 2014-02-09 20:42:11 +01:00
  • 3b0d3ad80e Release 1.68 Release_1.68 Michael Peter Christen 2014-02-09 17:37:35 +01:00
  • 195e5868d3 catch solr close exceptions Michael Peter Christen 2014-02-09 15:04:46 +01:00
  • 751c128544 extra sleep for remote searches enhances search results because there is more time for more remote peers to contribute on the first result page Michael Peter Christen 2014-02-09 14:57:17 +01:00
  • 0cabcbbe83 more efficient wordcount Michael Peter Christen 2014-02-09 14:45:12 +01:00
  • 3d474a843e added memory protection for postprocessing Michael Peter Christen 2014-02-09 12:36:56 +01:00
  • 412d55523c enhanced memory protection and OOM exception handling in Solr connector Michael Peter Christen 2014-02-09 12:36:14 +01:00
  • d9858e1b8a removed warnings and superfluous logging Michael Peter Christen 2014-02-09 12:26:58 +01:00
  • acc8d7faa7 fixed setting of shortMemoryStatus in MemoryControl Michael Peter Christen 2014-02-09 12:25:55 +01:00
  • 7e71dcc417 removed interaction fragments Michael Peter Christen 2014-02-09 12:25:07 +01:00
  • 94245ce0a8 fixed "Size in KBytes" calculation in PerformanceQueues_p.html, see http://bugs.yacy.net/view.php?id=362 Michael Peter Christen 2014-02-07 17:19:08 +01:00
  • 726e8c3ad5 removed unused classes and servlets Michael Peter Christen 2014-02-07 01:47:10 +01:00
  • 9f921c27f3 removed libraries which had been used by jena and were added together with jena in commit 5d20cd324a, see: 5d20cd324a Michael Peter Christen 2014-02-07 01:38:58 +01:00
  • 6e59ca4ebf removed jena library and all code that depended on jena. When jena was introduced, it was also used for search facets. The generic search facets are now deduced from generic solr fields which makes jena as tool for facet semantics superfluous. Michael Peter Christen 2014-02-07 01:20:06 +01:00
  • 0e6729f9bc Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git Michael Peter Christen 2014-02-07 00:23:50 +01:00
  • 9228214f9b enrichment of PerformanceMemory display of SolrInfoMBean table Michael Peter Christen 2014-02-07 00:22:31 +01:00
  • e8bdf16ea7 added statistic information for solr resources in PerformanceMemory Michael Peter Christen 2014-02-07 00:02:19 +01:00
  • 1a2b298a65 fix: select all checkbox Tables_p (needs form name attribute) reger 2014-02-06 23:15:00 +01:00
  • 931541d198 re-inserted default value re-set button to performance queues and patched missing values for recent new queues Michael Peter Christen 2014-02-06 22:39:19 +01:00
  • 8d60d4d56e Merge branch 'master' of gitorious.org:yacy/heronchatoyants-rc1 Michael Peter Christen 2014-02-06 21:25:03 +01:00
  • 05902e4e71 updated and completed translation of Steering.html and ConfigUpdate_p.html into French Alex Vouilloz 2014-02-06 20:57:07 +01:00
  • 456e52e0d5 enhanced strategy to clear solr caches - redesigned the instance mirror class (which was a mess) - added final method to close a searcher (which otherwise keeps a cache) - changed cache clear method which iterates over resources and calls clear to all caches in the searcher resources Michael Peter Christen 2014-02-06 19:13:29 +01:00
  • 0281d738d4 updated and completed translation of Network.html into French Alex Vouilloz 2014-02-06 00:46:14 +01:00
  • 48539f0a25 translated templates into French Alex Vouilloz 2014-02-05 19:22:59 +01:00
  • 52599a11b3 Merge branch 'master' of ssh://gitorious.org/yacy/rc1 Michael Peter Christen 2014-02-05 15:35:53 +01:00
  • bd1685c94a fix not needed getFileExtension().toLower (double) add missing .getFileExtension reger 2014-02-05 03:45:02 +01:00
  • ae6171aa6d completed fr.lng for Status.html Alexandre Vouilloz 2014-02-05 00:56:49 +01:00
  • 43d2cd4b47 completed fr.lng for BasicConfig.html (+modif in de.lng) Alexandre Vouilloz 2014-02-04 21:41:12 +01:00
  • a11f072504 enhanced didyoumean orbiter 2014-02-04 00:18:11 +01:00
  • c0e6a65ec3 enhanced didyoumean Michael Peter Christen 2014-02-03 18:49:03 +01:00
  • bc395c7439 reduced color depth of star icons (for smaller file sizes) Michael Peter Christen 2014-02-03 17:39:59 +01:00
  • 9e0e39a9a4 small change to start/stop/pause icon style Michael Peter Christen 2014-02-03 17:39:26 +01:00
  • 6d2dab7b21 fixed 'resource leak' warning Michael Peter Christen 2014-02-03 13:38:26 +01:00
  • 22e3524797 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git orbiter 2014-02-03 12:45:35 +01:00
  • c40ba51ca6 added new suggest method which replaces more-than-one suggestions: instead of computing suggest permutations of the given words, the completion of a phrase using the given words is searched in the fulltext index. orbiter 2014-02-03 12:44:52 +01:00
  • ad4b213145 remove unused static var from HTTPDProxyHandler reger 2014-02-02 03:47:12 +01:00
  • 6c6056836d fix vocabulary navigator checkbox selection (from last commit) reger 2014-01-31 23:03:01 +01:00
  • b693ce9759 allow combining selection of different search nav's (facets) - selecting more than one nav combines the 2 selections (with AND) - unselecting one nav clears all selected reger 2014-01-30 22:57:27 +01:00
  • cb71413d19 fix page nav, to keeping modifier (was new issue) reger 2014-01-30 22:00:32 +01:00
  • 416481c33e added a boost on appearance of combined words (in the same order the user submitted that) when searching for more than one word orbiter 2014-01-30 10:51:08 +01:00
  • 0b88137def Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git orbiter 2014-01-29 22:01:40 +01:00
  • c589ee8c6e URLproxy access check too tight respect config ip pattern (was own ip) reger 2014-01-28 22:39:45 +01:00