Mirrors/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2025-06-22 04:16:09 -04:00
Code Issues Packages Projects Releases Wiki Activity

Commit Graph

  • 11b4f80bde - fixed non-closing client connections - added client connection tracker in connections servelet orbiter 2007-09-25 21:36:08 +00:00
  • d352853f2d fix for non-closing client sessions orbiter 2007-09-24 08:42:07 +00:00
  • 1488769e1f cleanup of unmaintained and outdated performance methods: removed object pools in httpc. Object pooling is not recommended, if the creation of the object is not time-intensive. Object pools are only useful, if there is much computation necessary to create some basic data that is stored in the object pool and can be re-used. This does not apply to object pools in YaCy. Object pooling of client sessions would make sense if they would allow re-use of living connections to other yacy clients. But every connection is closed after usage of an object in the client pool, therefore the YaCy server client objects are not such that hold hardware/network-allocated entities. See: http://www.javaperformancetuning.com/news/qotm033.shtml http://java.sun.com/docs/hotspot/HotSpotFAQ.html#gc_pooling http://docs.sun.com/source/816-7159-10/pt_chap5.html http://www.microjava.com/articles/techtalk/recylcle2 orbiter 2007-09-23 20:49:52 +00:00
  • 3cb9cdc9be try to fix connection problem, possible cause for wrong junior status and non-passive passive peers: the YaCy client treats disconnections during data transmissions as error and discards all data transmitted so far this did not happen so far until I removed a delay time at the end of the daemon session which prevented this case. To fix this problem, disconnections during transmissions are not treated as error now, which means that end-of-transmissions with sudden disconnections are not a cause for peer diconnections any more. To be nice to non-updated peers, the sleep time at the end of server sessions is also re-enabled. orbiter 2007-09-23 17:31:29 +00:00
  • 00dab81077 simpler solution to last commit + works with and without navigation collumn on the left fuchsi 2007-09-20 01:52:10 +00:00
  • eb16a99e94 avoid floating of long page titles around the favicon in search results fuchsi 2007-09-19 22:08:56 +00:00
  • 9524b9c16a second try of rev 4100 :). Tested in Iceweasel/Firefox 2.0.6, Konqueror 3.5.7, Opera 9.23 (all linux) and IE6-SP1 (wine) fuchsi 2007-09-17 19:39:15 +00:00
  • 6b8faaadb6 undo last commit for further evaluation, a progressbar element is used on other pages as well... fuchsi 2007-09-17 03:36:35 +00:00
  • 1880bba420 A few changes to the progress bar and search result statistics layout influenced by the discussion in <http://forum.yacy-websuche.de/viewtopic.php?f=5&t=268> with the idea of saving vertical space. Please check in every available browser and comment wether it's better than before. ;) fuchsi 2007-09-16 14:30:53 +00:00
  • 404ebf1474 # update of de.lng - NO unused strings anymore!!! daburna 2007-09-16 10:17:22 +00:00
  • 041922652a # update of de.lng - removed or updated unsused strings - updated some files daburna 2007-09-15 13:10:56 +00:00
  • ba59de773f again and again junior - test borg-0300 2007-09-13 17:05:53 +00:00
  • 9fa75ef4d1 Limit the percentage of the progress indicator to reasonable values hermens 2007-09-13 16:37:23 +00:00
  • 4275727d69 fix for peer ping problem (implemented a 3-time re-ping); cause for 'Connection reset' still unknown orbiter 2007-09-12 00:42:53 +00:00
  • e78098be9b According to HTML-Specs "name" and "id" attributes share the same namespace. So we can't have one element with name="offset" and another one with id="offset". Additionally IE6's getElementById() returns elements with matching names as well and Opera is mimicing this behaviour. fuchsi 2007-09-11 16:21:14 +00:00
  • 07d1e98909 fixed round-robin method of peer-ping order (the successfully pinged peer was not updated to current last-seed date) orbiter 2007-09-11 16:07:35 +00:00
  • a1dcd065ad some tweaks to the search results layout fuchsi 2007-09-11 15:56:14 +00:00
  • 76e4c2d69e fix for peer-ping in case that remote peer does not respond with valid values orbiter 2007-09-11 15:27:01 +00:00
  • e192f99134 fix small bug introduced in r4089 that appeared when we tried to remove "gzip" encoding from Accept-Encodings header closes http://forum.yacy-websuche.de/viewtopic.php?f=6&t=336 fuchsi 2007-09-10 21:46:40 +00:00
  • ae4b9308ef Fix problems with some web servers which couldn't handle the way yacy was sending requests. Thx to celle for the patch. http://forum.yacy-websuche.de/viewtopic.php?f=5&t=320 fuchsi 2007-09-10 09:15:28 +00:00
  • 6601e37512 clear caches after changing blacklists, closes http://forum.yacy-websuche.de/viewtopic.php?f=6&t=241&p=1964#p1964 fuchsi 2007-09-10 08:15:25 +00:00
  • 5b0c1449e1 various fixes and cleanups for blacklist handling: 1. avoid adding duplicate file name entries in config properties for lists, 2. correctly merge all path masks from all list files for the same host masks, 3. rewrite helper methods standard java methods for Collection transformations, 4. merged various methods with identical functionality for different Collection implementations into one, 5. minor refactoring to improve code readability. fuchsi 2007-09-10 06:20:27 +00:00
  • e27aeb7fdc patch for bad crawl filter at crawl start orbiter 2007-09-09 19:21:41 +00:00
  • 841cf71022 fix for NPE in DHT transfer selection, see http://forum.yacy-websuche.de/viewtopic.php?f=6&t=327 orbiter 2007-09-09 19:08:13 +00:00
  • 3047ae2cd9 fixed some more old links to new hompage location orbiter 2007-09-09 18:43:39 +00:00
  • dbd1eeead5 fix for missing object miss-cache flush value: the value is alway zero because there is no miss-cache flush see http://forum.yacy-websuche.de/viewtopic.php?f=6&t=288 orbiter 2007-09-09 18:35:05 +00:00
  • f2a3434407 fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=238&p=1341#p1341 orbiter 2007-09-09 17:31:29 +00:00
  • 229ca2ba48 fixed/re-implemented rss-version of search result page orbiter 2007-09-09 12:30:18 +00:00
  • f4a5c287fe re-implemented post-ranking of search results (should enhanced search result quality) orbiter 2007-09-08 11:50:19 +00:00
  • d962200d11 added time format for better understanding daburna 2007-09-07 14:59:35 +00:00
  • 8ff5e2c283 - fixed/re-implemented media search - fixed search tipps (topwords, now appearing at the bottom of the page) - added search consequences execution (deletion of bad referenced some time after the search happened) - added some formatting at network table orbiter 2007-09-07 11:45:38 +00:00
  • 6c3bcadc1c - re-implemented image search - generalized search result status bar, is now also visible during text search orbiter 2007-09-06 13:26:38 +00:00
  • 6c819a6fd9 added cache to favicon display added better synchronization for simultanous search requests orbiter 2007-09-06 01:28:35 +00:00
  • d69013f66a added patch from Fuchs - http://forum.yacy-websuche.de/viewtopic.php?f=6&t=241 borg-0300 2007-09-05 11:51:02 +00:00
  • daf0f74361 joined anomic.net.URL, plasmaURL and url hash computation: search profiling showed, that a major amount of time is wasted by computing url hashes. The computation does an intranet-check, which needs a DNS lookup. This caused that each urlhash computation needed 100-200 milliseconds, which caused remote searches to delay at least 1 second more that necessary. The solution to this problem is to attach a URL hash to the URL data structure, because that means that the url hash value can be filled after retrieval of the URL from the database. The redesign of the url/urlhash management caused a major redesign of many parts of the software. Since some parts had been decided to be given up they had been removed during this change to avoid unnecessary maintenance of unused code. orbiter 2007-09-05 09:01:35 +00:00
  • 66905b7c97 update cause of the new searchpage daburna 2007-09-04 10:03:10 +00:00
  • e90afa9483 fixed search access tracker orbiter 2007-09-04 09:04:47 +00:00
  • 4779f314fe first version of next-generation search interface: - snippets are not fetched by browser using ajax, they are now fetched internally - YaCy-internat threads control existence of snippets and sort out bad results - search results are prepared using SSI includes - the search result page is visible right after the search request, the results drop in when they are detected - no more time-out strategy during search processes, results are shifted within queues when they arrive from remote peers - added result page switching! after the first 10 results, the next page can be retrieved - number of remote results is updated online on the result page as they drop in - removed old snippet servelet (which had been also a security leak btw) - media search is broken now, will be redesigned and fixed in another step orbiter 2007-09-03 23:43:55 +00:00
  • 34858be5ef added option to simple crawl start: complete domain crawl orbiter 2007-09-02 19:55:14 +00:00
  • d2360eaf68 - removed enctype="multipart/form-data" from the get-form of the peer-selection in Messages_p.html (in Konqueror this didn't work and multipart/form-data is only for post) - removed name="searchresults" from the searchresults (seems to be no longer needed and is invalid) - moved the favicons in the searchresults to the left side as requested in http://forum.yacy-websuche.de/viewtopic.php?f=5&t=268 - added alt-attributes for the favicons (images must have alt-attributes to be valid) michitux 2007-09-01 23:43:08 +00:00
  • 0e27febe47 *) fixed more links low012 2007-09-01 11:24:23 +00:00
  • 01ac8c8f6a *) fixed dead link low012 2007-09-01 09:15:55 +00:00
  • a493bd88b6 *) updated a few links low012 2007-08-29 16:19:00 +00:00
  • 6d759ad0a7 - new bot address - removed unused skins orbiter 2007-08-29 11:46:42 +00:00
  • 71e5d24f4a fix for watch crawler, see http://forum.yacy-websuche.de/viewtopic.php?p=1771#p1771 orbiter 2007-08-28 12:20:19 +00:00
  • f9e6cf6a3d more refactoring of search: integrated first version of ssi-using search interface, but the function is currently disabled orbiter 2007-08-28 12:15:46 +00:00
  • f81ef40cc4 no dht activity for small networks; this is not needed if the network is small orbiter 2007-08-26 22:35:26 +00:00
  • d9472b6a3a * fixed problem with watch crawler * added new column to network table (remote crawl urls): the new value for provided URLs will be used for new remote crawl method orbiter 2007-08-26 22:06:58 +00:00
  • e332b844b2 - enhanced remote search: during waiting time for remote crawls some urls are fetched so the url cache can be filled with these urls - the url-prefetch is used to sort out some unresolved urls - the snippet-fetcher is triggered with the search event id. This is used to remove missing snippets from the search cache so they will not be displayed again orbiter 2007-08-26 18:18:35 +00:00
  • a34d9b8609 * added a search history cache that maintains search results for 10 minutes it is necessary for the new search process that will do automatic re-searches a positive effect is, that when a re-search is done it can be monitored how many results had been contributed from other peers. The message for this contribution was moved from the end of the result page to the top. * enhanced re-search time when a global search was done an the local index has already a great number of results for this word * re-organised presearch computation; must be further enhanced orbiter 2007-08-24 23:12:59 +00:00
  • ae86d010bb more refactoring of search processes; also some small speed enhancements orbiter 2007-08-24 08:41:52 +00:00
  • b3c830271c fix in xml header orbiter 2007-08-23 16:58:40 +00:00
  • bb426565f0 added new yacy protocol for mass url-pull for better remote crawling distribution orbiter 2007-08-22 00:59:05 +00:00
  • 4f6d56330d Bugfix für abgeschnittene Überschriften - http://forum.yacy-websuche.de/viewtopic.php?f=6&t=273 borg-0300 2007-08-21 22:07:23 +00:00
  • 54004e929b *) Better Bourne-Shell (OpenSolaris) compatibility, update and restart really work now. As the Bourne-Shell is the grandfather of most modern shells, it should also work with Linux (tested with Mandriva, works) and OSX (Please test!). *) Fixed a typo. low012 2007-08-20 21:52:52 +00:00
  • 72752bb503 because of a new database structure handling, the memory need for accessing collection objects has been reduced to 50%: - set new memory calculation functions for indexing process - adjusted guessed memory amount -> Testing needed: try new recommended value (see performanceQueues) and see if OOMs occur. -> report maximum recommended value, so we can set new default values. orbiter 2007-08-20 17:36:43 +00:00
  • 9afd65bf82 small fixes: recommendation in performance queues and network unit domain orbiter 2007-08-20 17:23:07 +00:00
  • 0ad8499e66 - all parsers are activated by default for pro releases - slightly higher file size limits for parsers orbiter 2007-08-20 12:18:38 +00:00
  • 694defb257 *) better compatibility with OpenSolaris 5/07, updates should work now low012 2007-08-17 15:26:34 +00:00
  • 3ea42f34bd - fixed a layout-bug in MessageSend_p.html (for details see http://forum.yacy-websuche.de/viewtopic.php?f=6&t=255) - fixed two bugs with hasLayout/procentual widths in InternetExplorer in MessageSend_p.html and Messages_p.html michitux 2007-08-16 21:34:29 +00:00
  • 16c203f759 fixed remote search access tracker orbiter 2007-08-16 11:44:18 +00:00
  • 344911bfaa shorter minimum delay values for intranet crawl targets orbiter 2007-08-15 23:18:12 +00:00
  • f890cc86aa inserted forwarding patch from fuchs see http://forum.yacy-websuche.de/viewtopic.php?f=6&t=233 orbiter 2007-08-15 22:25:48 +00:00
  • b5346141b3 made the plasmaHTCache static (there is only one internet, so we need only one cache) orbiter 2007-08-15 21:31:31 +00:00
  • aa9a4c1dea #update of de.lng and fra.lng - fra: french translation taken out of the wiki - de: small cleanup: removed or updated unsused strings - de: added translation for CrawlProfileEditor_p.html and Supporter.html daburna 2007-08-15 20:51:51 +00:00
  • 947fc46904 refactoring of search process: - re-designed remote request result processing - re-designed local result accumulation, will be further enhanced with snippet fetcher - removed search process handling in switchboad - made snippet class static (there is no need for multiple snippet objects) - removed some redundant tasks in server-side search process, should be a little bit faster now orbiter 2007-08-15 11:36:59 +00:00
  • 3ca8f71cbb refactoring of dbtest to create separated kelondro sql connector interface orbiter 2007-08-11 22:40:24 +00:00
  • 5cf634a4a4 New media-search ui: - uses the progressbar introduced in the image-search - results are displayed using the same layout as the text-search - results are displayed in the order they arrive michitux 2007-08-11 22:20:01 +00:00
  • 61f93cbf14 some code-cleanup orbiter 2007-08-11 00:42:04 +00:00
  • e76e996737 fixed umlaute-problem orbiter 2007-08-10 14:10:57 +00:00
  • 4798044708 fixed compile problem with svn 4037 orbiter 2007-08-10 14:03:07 +00:00
  • 24e25e1141 enhanced SSI server-side support: - SSIs may now refer to servlets, not only files - calling a servlet, the servlet/SSI engine is called recursively - SSIs now work also for non-chunked-encoding supporting clients This will support the new search page functionality, to show search results dynamically without using javascript. To test this method, a test page has been added http://localhost:8080/ssitest.html ..calls dynamicalls 3 servlets, which produce some delays during their execution please verify that you can see the result step-by-step on your browser To implement this feature, some refactoring had been taken place, mostly code had been made static and will execute faster. orbiter 2007-08-09 21:58:38 +00:00
  • c8e5a4a6b7 *) fixed bug described by Huppi in http://forum.yacy-websuche.de/viewtopic.php?t=239 *) added a preview function to message system *) removed some old comments, I hope that's OK low012 2007-08-08 18:23:45 +00:00
  • f77898748b # update of de.lng - small cleanup: removed or updated unsused strings - added translation for IndexCreateWWWRemoteQueue_p.html daburna 2007-08-07 15:27:12 +00:00
  • 6dd674bb53 # oustanding update for German translation file daburna 2007-08-06 16:38:05 +00:00
  • 5c1b444690 some redesign of min/max and normalization computation during search result ordering this saves about 1 millisecond for each URL reference, which has some good effect on the search result computation if a word is searched that appears very often (speed-up of 1 second and more) orbiter 2007-08-06 12:50:11 +00:00
  • 9678d1b282 fixed new EcoRecords-Nodes. Here I omitted object content copying before to avoid massive System.arraycopy. That did obviously not protect enough the Node objects orbiter 2007-08-06 10:10:33 +00:00
  • 1af0e3bd84 refactoring orbiter 2007-08-06 00:56:56 +00:00
  • 5605887571 refactoring of search processes orbiter 2007-08-05 23:57:25 +00:00
  • 5dee7e9c29 *) addition to Rev 4028, see http://forum.yacy-websuche.de/viewtopic.php?f=6&t=204 low012 2007-08-04 08:26:53 +00:00
  • d3e777a98d bugfix for built-bug, see http://forum.yacy-websuche.de/viewtopic.php?f=6&t=204&hilit= orbiter 2007-08-04 00:44:15 +00:00
  • 62347b50f4 added security layer for ViewImage: - images may be requested by localhost and authorized users only, if the request is done using a clear-text URL - the image may be requested also using a code that can be a license to retrieve a URL for everyone - some servelets produce URL licenses for ViewImage, like image search results orbiter 2007-08-03 23:06:53 +00:00
  • 9a860cf397 bugfix for wrong record tracker message orbiter 2007-08-03 12:24:52 +00:00
  • 69d640b041 added missing synchronization in crawl balancer to avoid that the synchronization is triggered during many-time-used size() operation a notEmpty method was added that can avoid the synchronization many times orbiter 2007-08-03 12:21:46 +00:00
  • 9628db6cdc enhanced memory allocation during database access: - refactoring of kelondroRecords; this class is now divided into kelondroAbstractRecords, kelondroRecords, kelondroCachedRecords, kelondroHandle and kelondroNode - better abstraction of kelondroNodes, such nodes may now be crated by different classes - a new Node defining class kelondroEcoRecords defines Nodes that do not need so much allocation and System.arraycopy - there is less memory transfer on the bus, especially for collection index - now half of memory needed for web index access orbiter 2007-08-03 11:44:58 +00:00
  • 57a5b6fa71 some generalization of remote proxy configuration and setting handling in httpc orbiter 2007-08-02 00:42:37 +00:00
  • fac8de6e9b documentation update / release download location orbiter 2007-08-01 23:43:45 +00:00
  • ea960c2b61 release 0.54 orbiter 2007-08-01 23:22:21 +00:00
  • 367fc28928 corrected Brausse->Brausze orbiter 2007-08-01 22:15:51 +00:00
  • ea05c7d463 *) characters are saved in ASCII-safe notation in language data files now ('\u0063' instead of 'a'), this hopefully avoids lots of problems low012 2007-07-31 21:27:28 +00:00
  • 0339c65ee2 *) note to myself: Just because it works with Linux and most of it works with OpenSolaris does not mean the whole thing works with OpenSolaris. Being lazy never pays off! low012 2007-07-31 20:41:54 +00:00
  • 757703a938 synchronization of access tracker to avoid java-internal loop in TreeMap during shutdown see http://forum.yacy-websuche.de/viewtopic.php?p=1178#p1178 orbiter 2007-07-31 10:42:11 +00:00
  • beb2e062a7 fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=192&p=1129#p1129 orbiter 2007-07-31 10:17:39 +00:00
  • e76fe1c078 - replaced unicode characters in copyright holder name ('Brausse') - more logging for bootstrap seedlist loading - larger DHT chunks orbiter 2007-07-31 10:00:17 +00:00
  • 31023dbc7a added package declaration for language package orbiter 2007-07-30 22:53:51 +00:00
  • 255e754257 *) Removed unnecessary condition (always true). low012 2007-07-30 21:03:07 +00:00
  • a42d233944 *) Filename for output file is created from language code + .ldf now (e.g. en-GB.ldf) low012 2007-07-30 21:00:42 +00:00
  • ffc607d65d *) Fixed a small bug found by Lotus. low012 2007-07-30 13:03:10 +00:00
  • 33e2423bde *) Tool to find out the percentage of letters in a text. First step to build a tool which detects the language a text is written in. low012 2007-07-29 22:05:20 +00:00
  • 3ee9a26ef2 *) deleted unused program low012 2007-07-29 15:30:29 +00:00