Mirrors/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2025-07-21 09:04:37 -04:00
Code Issues Packages Projects Releases Wiki Activity

Commit Graph

  • 93c535d111 fixed http://forum.yacy-websuche.de/viewtopic.php?p=21113#p21113 fixed a concurrent modification exception during search and a time-out problem orbiter 2010-11-03 20:58:50 +00:00
  • 04932dc268 added rdf data structure for rss feeds orbiter 2010-11-03 20:06:23 +00:00
  • 84f2953cd8 fix for rss loader / rss type recognition orbiter 2010-11-03 19:58:01 +00:00
  • 4c72885cba added a sitemap entry parser and loader for sitemaps (a recursion if a sitemap refers to another sitemap) orbiter 2010-11-03 19:48:33 +00:00
  • 790e0b1894 - enhanced index deletion in IndexControlRWIs_p: delete also robots.txt database and cache if demanded - added option for details of deletion - added deletion to new ConfigHTCache_p servlet orbiter 2010-11-03 18:31:36 +00:00
  • 3e303db1fa added search suggestions in XML format. This also supports CORS orbiter 2010-11-03 09:20:36 +00:00
  • f5324b27f2 more updates to the new bookmarks (ymarks).... - split YMarkTables and YMarkIndex in two different classes - HTML import is working properly - XBEL import is still broken apfelmaennchen 2010-11-03 06:47:02 +00:00
  • 445619f3ec added a submenu ConfigHTCache_p.html to set the size of the HTCache separately from the proxy configuration. orbiter 2010-11-02 23:57:11 +00:00
  • 85c65475fa smal but important correction of last commit @ HTTPClient (if there is a response it really should be taken to its end) sixcooler 2010-11-02 21:18:44 +00:00
  • acd93b1b31 * add failsafe mechanisme to domainlist retrieval domainlist is saved locally, if none of the given urls in network.unit.domainlist could be retrieved, the file from the last boot is used instead f1ori 2010-11-02 17:57:48 +00:00
  • 70c95608d4 Added CORS Access header for yacysearch.rss output used some of the recommendations from Copro: http://forum.yacy-websuche.de/viewtopic.php?p=21015#p21015 Original Request: http://forum.yacy-websuche.de/viewtopic.php?p=20829#p20829 orbiter 2010-11-02 16:28:40 +00:00
  • 917d715374 lulabad found his signature orbiter 2010-11-01 23:32:20 +00:00
  • 18729351e7 upnp: hint for wrongly detected local ip address lotus 2010-10-31 20:05:59 +00:00
  • def4253555 * add option to network definition to provide a domainlist (syntax like in blacklists) * crawler and search allow only urls matching one in domainlist (if list is provided) * this may be useful to prevent dedicated networks from being "polluted" * FilterEngine is improved Backlist-object, Blacklist may inherit from FilterEngine in the future f1ori 2010-10-30 14:44:33 +00:00
  • 2db6ca8104 - do not allow user to disable java installation - java version update - build cleanup lotus 2010-10-30 08:49:39 +00:00
  • 482127e777 removed release key from location 2 because the signature of that source can not be verified. But the source is ok. orbiter 2010-10-29 09:22:56 +00:00
  • ac6b503adf untar files without gzip decompression even if the file has gz extension. this is done when the decompression fails. decompressed gzip files with gz extension may appear if the server sets a gzip compression header orbiter 2010-10-28 23:12:33 +00:00
  • efe0667fdd more new bookmark (ymarks) code with experimental html and xbel import apfelmaennchen 2010-10-28 15:24:15 +00:00
  • caabebf9be Fixed spelling mistake omiting -> omitting in debug messages in ConfigUpdate_p.java and Switchboard.java mikeworks 2010-10-28 04:03:11 +00:00
  • fb92f9ae8e added mime type image/jpeg (image/jpg is wrong but it is left here because it does not harm and this error also exists in configuration of web servers) see also: http://forum.yacy-websuche.de/viewtopic.php?p=21129#p21129 orbiter 2010-10-27 21:53:11 +00:00
  • 155d556568 - better memory protection - more logging - little bit of refactoring orbiter 2010-10-27 13:21:18 +00:00
  • a16e5e7a63 ConfigPortal.html: Added safe space character because empty <dt> produces a warning in HTML validator de.lng: Added translation for added ConfigPortal selection for target selection and WikiHelp changes mikeworks 2010-10-26 21:14:37 +00:00
  • 7d8de34778 * add a bit documentation to DigestURI, use DigestURI(string) instead of DigestURI(string, null) f1ori 2010-10-26 16:10:20 +00:00
  • 25a8e55bc9 more logging about bad seeds orbiter 2010-10-26 15:00:22 +00:00
  • 959b8c6fa0 - allow greater seed size - more logging for bad seeds orbiter 2010-10-26 14:40:19 +00:00
  • e103419a56 - removed <3 peers barrier for peer ping feedback - more logging orbiter 2010-10-26 13:08:09 +00:00
  • d0e6c03b51 some updates to the new bookmark code... apfelmaennchen 2010-10-25 22:44:05 +00:00
  • facfd204e9 added a parent configuration option. see /ConfigPortal.html requested here: http://forum.yacy-websuche.de/viewtopic.php?p=21099#p21099 orbiter 2010-10-25 22:16:07 +00:00
  • e3964f2c31 better catch of network definition load error; continue with secondary network load definition location orbiter 2010-10-25 09:20:45 +00:00
  • 7adfe4a1c1 fix for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=35#p21092 apfelmaennchen 2010-10-25 08:12:04 +00:00
  • b8d6784410 *) WikiCode help page update low012 2010-10-25 00:30:10 +00:00
  • 65a0381f76 *) cleaning up code (still not done) low012 2010-10-24 23:57:07 +00:00
  • e3e3b49d52 - enhanced main release recognition - yacybot user agent now includes the yacy network name (not the peer name!) - refactoring and clean-up (mostly turned tab into spaces) orbiter 2010-10-24 21:43:01 +00:00
  • 9c94ebdee4 small changes to new bookmark code... apfelmaennchen 2010-10-22 13:14:09 +00:00
  • 244b56e9d3 an update to the new bookmark code... apfelmaennchen 2010-10-21 19:18:17 +00:00
  • 96e625c541 AccessTracker: Fixed table output while no results are displayed --> all of AccessTracker should now validate XHTML 1.0 Strict, also tfoot must be defined before tbody Connections_p: Added static protocol "http" for incoming connections because only http pool is queried and -UNRESOLVED PATTERN- is not really useful de.lng: Minor changes and removed one old line mikeworks 2010-10-19 03:26:18 +00:00
  • dc40f51b8d *) added headlines as proposed by Vega *) <pre> will be displayed monospaced in wiki and blog again *) bugfix for <pre> spanning multiple lines *) replaced deprecated <s> tag with <span> equivalent low012 2010-10-18 23:00:43 +00:00
  • f035f257da added some more bookmark code... apfelmaennchen 2010-10-18 21:09:41 +00:00
  • 22ed9c380c *) fixed bug which was introduced in r7226 (shame on me) which made wiki unusable (all entries were stored with empty subject as key -> edits were lost) low012 2010-10-18 21:06:23 +00:00
  • 60fd2e549d * log failures when writing config file f1ori 2010-10-18 15:00:29 +00:00
  • 58e74282af added a word counter statistic in condenser which is used by the did-you-mean to calculate best matches for given search words. orbiter 2010-10-18 11:35:09 +00:00
  • 2a0eb09e08 enhanced html id names and tag cloud visualization orbiter 2010-10-18 09:36:49 +00:00
  • 863065abc4 added user agent logging to access tracker orbiter 2010-10-18 08:09:59 +00:00
  • 61e87c0b14 IndexControlRWIs_p.html, IndexControlURLs_p.html, ViewFile.html/.java: changes to HTML output and &nbsp; in case of empty values for XHTML strict / transitional validation de.lng: Added missing translation for Show Content and changed existing line --> Index Administration should now correctly validate XHTML 1.0 Strict / Trans mikeworks 2010-10-17 16:51:29 +00:00
  • a79728b97d some updates to experimental bookmark code... apfelmaennchen 2010-10-17 09:58:50 +00:00
  • ef782cd026 and even more experimental bookmark code... apfelmaennchen 2010-10-16 10:20:41 +00:00
  • ed4371dcf3 enhanced navigation implementation and enhanced tag cloud computation orbiter 2010-10-15 23:45:12 +00:00
  • ca738ac924 - added a tag cloud to search results (using the topics) - some refactoring of score classes - added default package for new classes add_ymark and delete_ymark orbiter 2010-10-15 22:01:39 +00:00
  • 7aca763ca8 Some more experimental bookmark code... apfelmaennchen 2010-10-15 12:53:41 +00:00
  • 4270ed696c Experimental code (I need to transfer the code to my macbook, sorry) for the new bookmarks API based on the Tables concept (same as for crawl starts). Currently you can add a bookmark by api/ymarks/add_ymark.xml?url=http://www.yacy.net&title=YaCy and watch the result via the standard view Tables_p.html. apfelmaennchen 2010-10-15 05:40:19 +00:00
  • e4d561971e added more score cluster options and made score cluster usage more transparent orbiter 2010-10-14 11:40:02 +00:00
  • e8f90201a5 fix for scheduling of rss feeds orbiter 2010-10-13 13:00:36 +00:00
  • 7cd9d9d22a - enhanced DidYouMean computation using a faster count on index entries; this causes that results can be ranked better - added limitations on DidYouMean result sets according to input and output string length orbiter 2010-10-12 22:02:10 +00:00
  • beb65437d2 additional fix for the widget - now a second result page is loaded automatically in case of too little search results for the scroll event to trigger apfelmaennchen 2010-10-12 21:55:24 +00:00
  • 2bb0c9b503 Fix for search widget keyup event handling. ESC will close the widget window and RIGHT will load additional search results, especially when the scroll event won't work because of too litte results. apfelmaennchen 2010-10-12 21:28:28 +00:00
  • de722090b5 enhancements in did-you-mean guessing orbiter 2010-10-12 09:45:15 +00:00
  • a59c885ee0 autocomplete and did-you-mean can now understand _all_ languages and can generate suggestions in all languages and character types orbiter 2010-10-12 08:36:33 +00:00
  • b7acd92ce4 Auto-Suggestions for YaCy Search: - added a suggest servlet according to opensearch and firefox standard - integrated the suggest servlet into opensearch description file - integrated a autocomplete plugin for jquery - added a autocomplete addition to the yacy search windows showing autosuggest queries orbiter 2010-10-12 01:23:49 +00:00
  • 67b3b4b13b de.lng: Added translation for skin color picker on http://demo.zuum.net:8080/ConfigAppearance_p.html ConfigAppearance_p.html: Some HTML 1.0 Strict changes on the Customization page http://demo.zuum.net:8080/ConfigAppearance_p.html --> Now all Customization pages should validate XHTML 1.0 strict mikeworks 2010-10-12 01:03:10 +00:00
  • 24f1cba7b2 performance hacks: - faster generation of index abstract compression during remote search - less synchronization in IO record reading - request index abstract generation only if necessary and faster time-out in remote search orbiter 2010-10-11 12:44:07 +00:00
  • 6a166c2040 patches for bad proxy behaviour - accept ipv6 localhost clients - index media files (url only) orbiter 2010-10-11 11:38:36 +00:00
  • d607b30b6a performance enhancements for search and code review for database functions - removed read cache from Records data structure because the read cache had no cache hit during search operation - copied old read-cache class to CachedRecords and the old, now new Records class does not have the cache any more and a code review checked that data structures and synchronization is clean - removed unnecessary synchronization from Table class during get() orbiter 2010-10-11 11:01:50 +00:00
  • 6d61b80fb6 added ColorPicker to WatchWebStructure orbiter 2010-10-11 08:57:40 +00:00
  • 45b1ab3d07 custom + generic skins: - added a generic skin which is filled with actual color assignment using a servlet - enabled css servlets - added a generic color scheme in configuration file - added configuration input in Customization/Appearance servlet - added a jquery color picker widget - placed color picked widget to input field of generic colour definition input fields orbiter 2010-10-11 00:00:10 +00:00
  • 41bf8ef9f9 de.lng: Added translation for Link List option in Crawl Site page http://localhost:8080/CrawlStartSite_p.html mikeworks 2010-10-10 10:44:15 +00:00
  • fcd40cd30f - disabled domZones (buggy, must think about better solution) - increased time-out for dns resolver and isLocal property orbiter 2010-10-09 10:17:50 +00:00
  • ec38eca278 fix for new URI equal method orbiter 2010-10-09 09:27:31 +00:00
  • 0d363a94d7 more performance hacks this makes YaCy search results VERY fast for all verify=false search cases and it enhances the search speed also for all other snippet-fetch cases. With this change my peer performed 100 Queries Per Second (!!!) while doing 10 queries simultanously (!!!) in an intranet index of 20000 URLs on my 16-core Mac orbiter 2010-10-09 08:55:57 +00:00
  • b8aee6d402 performance hacks for better search performance orbiter 2010-10-08 23:50:28 +00:00
  • 11bebe356b fixed crawl start: with SVN 7225 the name of the crawl start url was not given in input field and therefore all crawl starts had contained the empty string as crawl start url orbiter 2010-10-08 22:02:24 +00:00
  • 2971c91988 fix for http://forum.yacy-websuche.de/viewtopic.php?p=20977#p20977 orbiter 2010-10-08 21:44:28 +00:00
  • 091dd3f6ec - enhanced intranet search speed - enhanced intranet portscan speed (better time-out) orbiter 2010-10-08 10:54:13 +00:00
  • b9f405d1e8 *) added comments *) more beautyful and easier to understand code (IMO) *) added display= parameter to a lot of links in Wiki.html low012 2010-10-08 00:32:50 +00:00
  • 70576e88d2 de.lng: Added some more untranslated strings I found and uncommented old ones that were removed terminal_p.html: Put back the old ID which was really easy to find IndexCreate.js: Because XHTML 1.0 Strict does not allow name tags for some elements rewrote most element access functions to use getElementById Table_API_p.html and all other html pages: Some XHTMl 1.0 Strict fixes, changed checkAll javascript, marked the first row with checkboxes as unsortable where applicable Table_API_p.java and all other java pages: URLencoded lines with possible ampersands & -> &amp; for validation XHTML 1.0 Strict sourcecode --> All Index Create pages should validate now. Hope I did not break anything else (too much :-) mikeworks 2010-10-06 00:00:23 +00:00
  • efa59250f8 release 0.98 for SuMa-eV Demo tomorrow orbiter 2010-10-05 17:52:54 +00:00
  • 6e6994e328 latest bugfixes to search and indexing function after test of demo presentation orbiter 2010-10-05 17:49:53 +00:00
  • c3bf17a3a1 fixed must-match filter for smb crawling orbiter 2010-10-05 00:05:08 +00:00
  • 099def2a04 small changes in search widget appearance orbiter 2010-10-04 23:43:33 +00:00
  • 50586a0dfd rename of widget to 'widget' orbiter 2010-10-04 19:57:29 +00:00
  • dffa142529 Fix for author navigator in yacyui-portalsearch.js apfelmaennchen 2010-10-04 19:27:12 +00:00
  • 574346f8ce better must-match pattern for intranet file-crawls orbiter 2010-10-04 12:55:39 +00:00
  • aacf572a26 - enhancements for search speed - bug fixes in many classes including basic data structure classes orbiter 2010-10-04 11:54:48 +00:00
  • aa6075402a smal fix for crawling from 'sitelist' at changes from 7214 sixcooler 2010-10-01 22:41:28 +00:00
  • 61c82f3105 gzip-compresson @ transferRWI & transferURL back again This reduce upload-volume to suit limited bandwidth of home-users like me :-) sixcooler 2010-10-01 00:42:43 +00:00
  • 2c549ae341 fixed a number of small bugs: - better crawl star for files paths and smb paths - added time-out wrapper for dns resolving and reverse resolving to prevent blockings - fixed intranet scanner result list check boxes - prevented htcache usage in case of file and smb crawling (not necessary, documents are locally available) - fixed rss feed loader - fixes sitemap loader which had not been restricted to single files (crawl-depth must be zero) - clearing of crawl result lists when a network switch was done - higher maximum file size for crawler orbiter 2010-09-30 23:57:58 +00:00
  • f6eebb6f99 replaced auto-dom filter with easy-to-understand Site Link-List crawler option - nobody understand the auto-dom filter without a lenghtly introduction about the function of a crawler - nobody ever used the auto-dom filter other than with a crawl depth of 1 - the auto-dom filter was buggy since the filter did not survive a restart and then a search index contained waste - the function of the auto-dom filter was in fact to just load a link list from the given start url and then start separate crawls for all these urls restricted by their domain - the new Site Link-List option shows the target urls in real-time during input of the start url (like the robots check) and gives a transparent feed-back what it does before it can be used - the new option also fits into the easy site-crawl start menu orbiter 2010-09-30 12:50:34 +00:00
  • 63e387508c ConfigLanguage_p.java: Fixed the filename for the API call to ConfigLanguage_p.html - previously ConfigLanguage.html was recorded and the action could not be replayed with error 404 - Not found mikeworks 2010-09-30 03:16:17 +00:00
  • f468d377d7 Collage.html and Collage.java: Added <p> in body before images for XHTML 1.0 Transitional validationg and alt tag to images as well as closing tag <img (...) /> terminal_p.html: Set new link for starting a crawl to CrawlStartSite_p.html and replaced the old embed object of the Among.us Flash object by their new JS which takes care of adding the object correctly de.lng: Moved the translations for the JS part from yacyinteractive.html to the yacyinteractive.js part --> Terminal page is now valid XHTML 1.0 Transitional mikeworks 2010-09-29 23:11:29 +00:00
  • 3057a0b939 - intranet scanner now produces urls with host names, not ips if possible - CrawStartIntranet servlet shows IPs and host names orbiter 2010-09-29 22:44:49 +00:00
  • 75964909aa added missing path to htroot (may only be necessary for cross-linking of servlet classes) orbiter 2010-09-29 22:19:41 +00:00
  • 421aa6a8bb ConfigLiveSearch.html: Fixed some HTML problems to validate at least XHTML 1.0 Transitional - strict is not possible because iframes are used. Replacing iframes with embedded object tag does not work in IE ConfigPortal.html: Fixed some HTML problems to validate at least XHTML 1.0 Transitional - for strict the target attribute of the a link has to be removed yacyinteractive.html: Moved all JS code to an external yacyinteractive.js file in JS folder yacysearch.html: Removed embedded scripts from in between the body tags - now everything is loaded in the header de.lng: Just in case JS files will be parsed at some point added translation for yacyinteractive.html result counter mikeworks 2010-09-29 20:06:39 +00:00
  • c60aed4435 no caching in browser of dynamic web pages sent by YaCy http this may prevent unnecessary IO caused by cache storage of the browser orbiter 2010-09-29 19:56:42 +00:00
  • 41a93ff565 de.lng: Small changes to the German translation and added missing part in Surftips.html and YaCy statistics link in left menu in header.template mikeworks 2010-09-29 06:08:19 +00:00
  • b7bb0cabaf Blacklist_p.html: Minor HTML and Javascript changes to get XHTML 1.0 Strict validation, lowercae onchange, id tags instead of name tags mikeworks 2010-09-29 05:41:32 +00:00
  • cd505d7e30 de.lng: German translation of the new Intranet Servlet introduced in SVN 7203 in CrawlStartIntranet_p.html CrawlStartIntranet_p.html: New Intranet Crawl Start Servlet - minor HTML changes to get XHTML 1.0 Strict validation, remove (double) name tags, remove single ending </dt> mikeworks 2010-09-28 22:05:12 +00:00
  • e63896f2a8 added an intranet scanner and a servlet which shows all intranet addresses and an option to start a site-crawl for all these addresses at once. orbiter 2010-09-28 12:18:54 +00:00
  • e54cb7fb0c more bugfixes (also for latest commit) orbiter 2010-09-28 10:20:46 +00:00
  • be6b48311c misc bugfixes orbiter 2010-09-28 10:00:33 +00:00
  • 44874f2cb9 Added "encoding =" UTF-8 "in the RSS files suessthomas 2010-09-27 20:36:49 +00:00
  • d2fd93135c - moved yacybot user agent string definition to MultiProtocolURI since there are basic access mechanisms where the bot string is needed - migrated the 'yacy' user agent to 'yacybot' in many client methods since the 'yacy' user agent is only used for the proxy orbiter 2010-09-27 14:54:32 +00:00