Commit Graph

  • bb2e03761c - fix for deadlock with 100% CPU during search - fix for failure of ranking because of a ConcurrentModificationException orbiter 2010-01-07 12:41:43 +00:00
  • 3f771d2a16 fix for rss parser: be lazy when rss is not well-formed orbiter 2010-01-07 01:02:23 +00:00
  • dff4f95c78 some patches to get the torrent parser working orbiter 2010-01-07 00:42:12 +00:00
  • 75dfe4098c Clean up the de.language File, removed all unused Blocks. All unused Blocks copied in a new File and archived this on my Homepage (http://tokeek.de). The Cleanup de.language File is easy readable, is the basic for more improvements. suessthomas 2010-01-05 21:24:36 +00:00
  • 574f49903e Prevent blob merge from possibly losing the last container hermens 2010-01-05 01:34:46 +00:00
  • 83d05e9176 added sixcoolers hack with some modifications: http://forum.yacy-websuche.de/viewtopic.php?p=15004#p15004 old index blobs where deletions have been made because of DHT transmission should be melted down to new blobs. This uses sixcoolers methods from the forum thread but modifies the process in such a way that the blobs are not merged with themselves but simply rewritten to smaller files. orbiter 2010-01-04 18:12:03 +00:00
  • fbd24c2d84 integrated the torrent parser orbiter 2010-01-04 16:07:31 +00:00
  • bd32f8b8cb added a torrent metadata file parser orbiter 2010-01-03 23:26:15 +00:00
  • 610e3ffffb Added new classes for the implementation of concurrent greedy algorithms. These classes can be used to produce an abstract worker process that can be used for common problems in artificial intelligence, such as game playing and problem solving. These classes will be used as abstraction layer for a new search process in YaCy. These classes had been created while searching for an abstraction of the current search process. It turned out that the abstraction of the YaCy search process is also an abstraction for problems in artificial intelligence and therefore the classes had been designed in such a way that it covers not only the YaCy-specific problem but also the more generic problems in ai. To test the classes they had been used in a ConnectFour implementation (game playing). orbiter 2010-01-03 22:51:14 +00:00
  • d0b7bf9ca2 added a decoder class for Bencoding orbiter 2010-01-03 22:44:09 +00:00
  • 04d91c67d6 *) added message if unrestricted access from localhost is possible *) added link to admin access config page low012 2009-12-28 14:01:41 +00:00
  • 028657f019 *) adding more SVN properties *) minor changes low012 2009-12-28 13:11:07 +00:00
  • 82d740050f *) adding more SVN properties *) minor changes low012 2009-12-28 12:26:50 +00:00
  • e04cb8cef0 *) adding more SVN properties *) minor changes low012 2009-12-28 12:16:40 +00:00
  • dcb1096fb0 *) adding more SVN properties low012 2009-12-28 11:22:32 +00:00
  • 7d610e0063 *) minor changes low012 2009-12-28 11:20:34 +00:00
  • 82198acc06 *) minor changes low012 2009-12-28 11:06:49 +00:00
  • b75547fc60 *) minor changes low012 2009-12-28 11:05:50 +00:00
  • e8fb332567 *) script does not allow empty passwords anymore low012 2009-12-28 10:16:39 +00:00
  • d933763b12 Small Changes on the de. language File and small correction to the ConfigAppearance_p.html page. Please test and report bugs to me (th-suess@gmx.de). suessthomas 2009-12-26 21:18:20 +00:00
  • 9bee0ac780 more logging for DHTrule lotus 2009-12-21 14:02:00 +00:00
  • 57d729e377 fix for negative numbers in network statistic orbiter 2009-12-21 11:36:48 +00:00
  • f8dc1dfabe include query in document title lotus 2009-12-19 21:26:19 +00:00
  • 4ac4fe952c patch for npe in bookmarks orbiter 2009-12-18 10:14:05 +00:00
  • c14233a933 fix for a OOM in MapView that can cause unavailability of - seed list - bookmarks during very low memory configuration orbiter 2009-12-15 12:23:37 +00:00
  • d548bd41ad fix for a npe during search orbiter 2009-12-15 12:15:39 +00:00
  • 37245430c3 fix for NPE during DHT RWI selection orbiter 2009-12-15 00:02:10 +00:00
  • 959b38b61b fix for memory tracker orbiter 2009-12-14 20:23:11 +00:00
  • bf08e59334 Conversion of the de. Language File into UTF-8 format. Please test and report bugs to me (th-suess@gmx.de). Small Changes on the de. language File. The old Language File archived on http://tokeek.de suessthomas 2009-12-13 22:11:03 +00:00
  • a37878b7d5 url parser regex performance hack orbiter 2009-12-10 14:40:32 +00:00
  • 2126dffea2 fix for rss/xml output for image search from AndreasB orbiter 2009-12-09 23:58:56 +00:00
  • b527d2ebfa fix for media search orbiter 2009-12-09 23:47:37 +00:00
  • 362b7a929b added extensive memory protection logic to avoid out of memory errors that may be caused by the RowCollection memory allocation function orbiter 2009-12-09 23:27:26 +00:00
  • 8281e29963 - more configuration for profiling graph (number of events) - more logging for a shutdown: print reason and accessing IP into log orbiter 2009-12-08 14:25:51 +00:00
  • 5f0f6b71b4 * revert last commit, something is more broken than before * UTC timestamps and lastseen-properteries still needs some debugging f1ori 2009-12-06 21:54:32 +00:00
  • 8c8b642eba * fix timezone problem f1ori 2009-12-06 21:03:12 +00:00
  • 713cb26a27 update for memory observer algorithm disable dht if memory is less than treshold after 4 times, maximum 11 minutes between each detection lotus 2009-12-06 17:45:48 +00:00
  • a44112b562 - moved index cleaner to blacklist submenu, because the index cleaner cleans the index with the blacklist - version switch to 0.93 to reflect advancements orbiter 2009-12-03 14:26:30 +00:00
  • 4782d2c438 fix for search bug that appeared when looking at page 3 of results or further orbiter 2009-12-03 12:25:03 +00:00
  • 29fde9ed49 better control of ranking order in sort stack orbiter 2009-12-03 00:36:07 +00:00
  • 93caa38d55 fix for bug in SortStack (did not appear to shrink according to required size) - caused bad and unsufficient search results orbiter 2009-12-03 00:20:53 +00:00
  • 66923ebc6c - modified method in RequestHeader that delivers the host name of requester: no more reverse domain lookup (may have killed interface performance in some cases) - added logging output for shutdown servlet: show ip of requester of the shutdown orbiter 2009-12-02 22:56:28 +00:00
  • e34e63a039 preset of proper HashMap dimensions: should prevent re-hashing and increase performance orbiter 2009-12-02 14:01:19 +00:00
  • 4a5100789f replaced _all_ size() == 0 with isEmpty() and all size() > 0 with !isEmpty(). The isEmpty() method is much faster in some cases, especially when used to access badly balanced hashtables where an size() operation becomes a large iteration. orbiter 2009-12-02 00:37:59 +00:00
  • f4946eaf27 - better thread dump - suppressed one server exception orbiter 2009-12-01 22:53:36 +00:00
  • 9743b70d1c disabled keep-alive of server, not really needed for speed but a cause for much trouble and memory occupancy orbiter 2009-12-01 19:14:16 +00:00
  • a4e72928ad classpath correction lotus 2009-11-29 19:56:25 +00:00
  • 173aa84611 *) Ooops! low012 2009-11-29 19:37:49 +00:00
  • e47e8a8bfe *) corrected classpath settings low012 2009-11-29 19:36:15 +00:00
  • 491ba6a1ba - some refactoring in workflow - some refactoring in search process - fixed image search for json and rss output - search navigation on bottom of search result page in cases where there are more than 6 results on page - fixes for number of displayed documents - disabled pseudostemming orbiter 2009-11-24 11:13:11 +00:00
  • 969123385b added json and rss output for image search orbiter 2009-11-23 16:10:50 +00:00
  • 016fdfa310 stub for json image search (not yet ready) orbiter 2009-11-23 00:43:51 +00:00
  • 82258b2882 default skin: rounded fieldsets lotus 2009-11-21 12:42:59 +00:00
  • 40900eebb4 *) better comment *) r6500! low012 2009-11-20 21:53:43 +00:00
  • 19fd5a3c33 *) script is much more userfriendly now low012 2009-11-20 21:49:39 +00:00
  • d183f8d980 refactoring (moved code from ContentTransformer to TemplateEngine) orbiter 2009-11-20 14:57:00 +00:00
  • 23aef43786 - better synchronization in SortStack - better ThreadGroup organization - less worker threads for media search (64 was too much...) orbiter 2009-11-20 14:35:33 +00:00
  • 7b1f5b0430 - better media search ranking - better concurrency with enhanced synchronization in sort stack orbiter 2009-11-20 13:19:12 +00:00
  • 4df88a4e7a - fixes for missing or bad hashCode computation - fixes for bad equals() methods that had not been used by hash maps and therefore some classes did not work as objects in hash maps. - this may also affect some cases where double-checks should have been, but did not work. orbiter 2009-11-20 12:11:56 +00:00
  • dbdf2570ba added comparator and more fixes for SortStack/SortStore orbiter 2009-11-20 03:30:48 +00:00
  • d2938c44a1 - added bmp parser to the document parsers - image parser that implement the document parser interface return itself in the list of images of the document which should cause that the parsed images contribute to the image search orbiter 2009-11-19 23:22:53 +00:00
  • 1dff620181 Better implementation of SortStack and SortStore and adoptions in all using classes to implement the necessary Comparable interface and hash code computation. The better SortStack performance affects crawling and image search speed and quality. orbiter 2009-11-19 13:49:28 +00:00
  • fe41a84330 some enhancements in web caching: avoid double loading of response metadata and/or content orbiter 2009-11-19 10:17:26 +00:00
  • 06d0dcde20 more enhancements to image search orbiter 2009-11-19 00:43:42 +00:00
  • 4c6312d103 enhanced image search orbiter 2009-11-18 23:56:05 +00:00
  • 2d8f3ee301 some performance hacks orbiter 2009-11-18 16:03:28 +00:00
  • 36fbfdcb21 more performance for remote search orbiter 2009-11-18 15:13:06 +00:00
  • 5c7b32a4fa better performance for list api (blacklist transfer) orbiter 2009-11-18 15:11:52 +00:00
  • 94b2a664f3 - use a static DiskFileItemFactory (one instantiation is enough) - use more memory for the DiskFileItemFactory to avoid IO when POST commands come orbiter 2009-11-18 15:05:51 +00:00
  • 267108470f testing jmx console for yacy: - start YaCy with startYACY.sh -l - open jconsole and open localhost:9999 orbiter 2009-11-18 14:54:30 +00:00
  • d9835d8568 Conversion of the de. Language to UTF-8 format. This version will replace the now used language File in the future. Please test and report bugs to me (th-suess@gmx.de). suessthomas 2009-11-17 16:19:46 +00:00
  • fd0658ce7c avoid forced execution of InetAddress.getLocalHost() at startup, because that hangs at some strangely declared linux configurations. The Domains.localHostAddresses object is first instantiated with a more simple logic and enriched with more host addresses using a concurrent thread that will not block a startup process. orbiter 2009-11-16 23:08:20 +00:00
  • 013f337d3f - avoid unnecessary host name lookups for localhost - avoid unnecessary reverse domain name lookups for remote access orbiter 2009-11-16 23:00:54 +00:00
  • 141712ec95 *) small changes to UI *) password will not be deleted anymore when changing to unlimited access from localhost low012 2009-11-15 22:15:25 +00:00
  • 12dd8ece3e enabled memory protection from 6459 with 50000kb (disables dht-in) this should only apply if there is really little memory available because it is checked by threads explictly requesting memory lotus 2009-11-13 16:26:45 +00:00
  • 20c5d78a5c fix for a ConcurrentModificationException orbiter 2009-11-11 23:31:12 +00:00
  • 5afd9f7a91 fix for crlf writing orbiter 2009-11-11 22:50:44 +00:00
  • 7144d2df6e added crawlReceipt servlet as individual class to examine OOM problem as documented in http://forum.yacy-websuche.de/viewtopic.php?p=18120#p18120 orbiter 2009-11-11 16:12:00 +00:00
  • 2d3c98b742 less computation within synchronized blocks orbiter 2009-11-11 16:07:40 +00:00
  • 1a146b0d73 added a patch to ignore bad mime-ignore patterns orbiter 2009-11-11 15:49:53 +00:00
  • 29fe436e36 - fixed post-ranking including prefer mask - enhanced a core database access method / less wasted ram orbiter 2009-11-09 19:14:51 +00:00
  • e9ab130ad7 fixed start/stop using ant orbiter 2009-11-09 19:12:33 +00:00
  • 5399d1e2bc refactoring (reason: get more abstraction to use the blacklist class; for integration in other servlets) orbiter 2009-11-08 22:58:57 +00:00
  • a97fdb4566 catch for NPE in image parser orbiter 2009-11-07 23:39:31 +00:00
  • 9ee7862710 *) added configuration script low012 2009-11-07 13:21:27 +00:00
  • 534182559c removed concurrency hacks from SplitTable because it showed deadlock-like situation. see thread dump at http://forum.yacy-websuche.de/viewtopic.php?p=18081#p18081 orbiter 2009-11-07 11:52:03 +00:00
  • 1fa0ac26e9 better protection against NPEs during search/ranking orbiter 2009-11-07 10:58:33 +00:00
  • 2bab0679e0 lost my key :-( orbiter 2009-11-06 23:46:29 +00:00
  • 4c99d4683d possible fix for lost crawl profile handles: clean-up job did wrong measurement to see if crawl is still running. orbiter 2009-11-06 23:15:20 +00:00
  • cd6745b292 accept rss feeds without channel descriptions orbiter 2009-11-06 22:46:21 +00:00
  • 08f1cbb125 another update to the pdf parser orbiter 2009-11-06 22:41:37 +00:00
  • 54c54fb144 get a handle for grep: 'StackTrace' orbiter 2009-11-06 19:55:21 +00:00
  • 605e896d6c more details for exception catching when parsing pdfs orbiter 2009-11-06 19:47:24 +00:00
  • 18b21eaffe small fixes to search default values and server logging orbiter 2009-11-06 19:13:35 +00:00
  • 6edc168cfe option to disable dht by memory limit: memory.acceptDHT in kbytes not yet pre-enabled, will clear on every startup please review since this could break dht in freeworld lotus 2009-11-06 19:13:30 +00:00
  • 4431b9767e added about 450 replacements for printStackTrace() methods to pipe such traces into the log at DATA/LOG/ orbiter 2009-11-05 20:28:37 +00:00
  • e3025ee691 - new icon for OAI-PMH loading action - added many stack trace outputs for exceptions in crawl profile handler to find the 'missing profile handle' bug - catched one more timeout exception in httpd file loader orbiter 2009-11-05 16:40:15 +00:00
  • f0b8db93f0 - more abstraction of serverCore thread access - no more keep-alive when number of connections exceeds 1/2 of the allowed number of connection orbiter 2009-11-05 14:54:43 +00:00
  • 19f31bb043 - moved OAI-PMH source list file from SETTINGS to DICTIONARIES/harvesting - added convenience method for loading of files from the web in LoaderDispatcher orbiter 2009-11-04 16:15:28 +00:00
  • 2889b9426e missing code for last commit orbiter 2009-11-04 12:03:19 +00:00