Mirrors/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2025-07-21 09:04:37 -04:00
Code Issues Packages Projects Releases Wiki Activity

Commit Graph

  • 15d9ea8375 *) changes for better code readability low012 2010-04-13 01:25:15 +00:00
  • 2bc459252e *) changes for better code readability low012 2010-04-13 01:16:09 +00:00
  • dc93cec3a8 *) Java 1.5 compatibility (see http://forum.yacy-websuche.de/viewtopic.php?f=8&t=2764) low012 2010-04-13 00:25:46 +00:00
  • 67ec58d8e7 search performance enhancement orbiter 2010-04-12 07:31:43 +00:00
  • 4ec0092677 more null == proxy fixes hermens 2010-04-10 18:31:12 +00:00
  • 2f90f0ad56 Remove asserts blocking proxy use cases hermens 2010-04-10 15:12:39 +00:00
  • ef467a0303 Another workaround for the second part of http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2770 This should prevent URLs with bad referrer entries from being dropped by transferURL or even crashing the whole Transmission$Chunk hermens 2010-04-10 13:57:46 +00:00
  • eb2a4bb555 workaround(?) for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2770&start=0&st=0&sk=t&sd=a&hilit=DefaultCharsetStringPart sixcooler 2010-04-10 00:21:07 +00:00
  • 25aef069a6 continuing String-hash - to - byte[]-hash redesign that was started in SVN 6775 orbiter 2010-04-08 00:11:32 +00:00
  • aad0a3045a Fixed CSS file yacyui-portalsearch.cs border-width: small; --> border-width: thin; margin-botton --> margin-bottom Forum Thread: http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2767 mikeworks 2010-04-06 00:04:01 +00:00
  • 6ef4900ec5 Changed function name status() -> statuscheck() in yacyui-portalsearch.js fixing the "status is not a function" bug visible in Firefox error console: http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2625 mikeworks 2010-04-05 12:44:35 +00:00
  • b97ad0f380 *) some minor changes for better code readability *) added more SVN properties low012 2010-04-05 12:37:33 +00:00
  • 7a3c19846f Updated German translation de.lng: added new Table_RobotsTxt_p.html and some other changes Changed 'Sprache' -> 'Language' in yacydoc.html and added translation in de.lng mikeworks 2010-03-29 00:56:51 +00:00
  • ba51d140e1 added more info in assert in balancer orbiter 2010-03-28 22:59:19 +00:00
  • 000dad034b *) better usage of JS timers (intervals) *) added messages when status is checked low012 2010-03-28 22:17:36 +00:00
  • a85c5bb8a7 added support for multiple (fail-over) network definition locations when http-locations are given. multiple locations can be given with a comma-separated list of urls pointing to the network definition file orbiter 2010-03-27 23:15:15 +00:00
  • f83b1b91b9 increased dht busy sleep time to 10 seconds to reduce TCP/IP traffic for default settings. 2 seconds had been too much traffic for home-use routers. Please try to set your dht busy sleep time in existing installations also to 10 seconds. orbiter 2010-03-27 23:04:00 +00:00
  • 9b3840cb66 performance hacks for the template engine + cleanup orbiter 2010-03-27 22:52:48 +00:00
  • 5c10f8bc5f enhanced latest hack orbiter 2010-03-27 07:19:49 +00:00
  • b3238bec83 performance hack for httpd orbiter 2010-03-27 07:09:55 +00:00
  • 1e8e79b9ef redesign of reference hash (URL-hash) parameter hand-over: pass value as byte[], not as String. This should cause that less byte[] <-> String conversions are made during time-critical tasks. This redesign is not yet complete, more to come .. orbiter 2010-03-26 18:33:20 +00:00
  • 72d8e9897b removed unnecessary cache flush call in backend of BufferedRecords orbiter 2010-03-26 12:44:13 +00:00
  • 749ffbd642 - added another catch case for the index dump and index merge process that should cause non-blocking behavior in case that index dump and/or index merge caused any unexpected exception. - reverted SVN 6766, this is too dangerous (may cause unexpected memory usage) and should not be necessary orbiter 2010-03-26 10:46:40 +00:00
  • 9ddb8e4a43 set an option for the java-internal image parser that prevents that the image is cached using the file-system in a temporary file. This should speed up image parsing during image indexing dramatically and should also cause better performance when showing the yacy banner and OSM tiles. orbiter 2010-03-26 10:43:31 +00:00
  • 312ca5d917 removed flush at end of every rwi entry since this reduces the write performance. This should speed up RWI cache dump and RWI merge operations and should cause less blocking time during these processes for the indexer. orbiter 2010-03-26 10:41:20 +00:00
  • 0018163c07 moved table row/column matching method from front-end to back-end orbiter 2010-03-26 10:01:27 +00:00
  • e12f1fd821 - added setting of access rights for executable scripts after auto-installation The correct access right was missing expecially for bin/apicall.sh orbiter 2010-03-25 09:51:01 +00:00
  • 21fcbcc35f added sorting function in network table, reverting SVN 6736 (not removing new sorttable) orbiter 2010-03-25 07:47:41 +00:00
  • 31e29a8831 - removed synchronization during index dump and index cleaning - added semaphores to synchronize index dump and index cleaning for each process separately orbiter 2010-03-25 07:09:53 +00:00
  • 95f31da8da increase dump cache queue length from 1 to 2 orbiter 2010-03-24 20:36:35 +00:00
  • fad3abb524 Tables_p.html servlet can now show tables with selected rows using a search field orbiter 2010-03-24 10:52:37 +00:00
  • 4c6dc396d8 *) more beautyful (IMO) code, no functional changes low012 2010-03-23 21:05:26 +00:00
  • 6c093d6aed - enhanced domain navigator computation - fixed domain navigator content in case that a mustmatch constraint was given orbiter 2010-03-23 13:41:41 +00:00
  • bb63c5d075 using a Pattern object with precompiled regular expressions to apply must-match constraints to search results: should speed up pre-sorting of search results and should cause richer search result sets orbiter 2010-03-23 10:17:28 +00:00
  • 5233177a7f A small typo fixed suessthomas 2010-03-23 08:44:34 +00:00
  • e0da0a84b0 performance fix in http parser orbiter 2010-03-22 09:12:52 +00:00
  • 90dd197ae7 - no latency for local crawls - catch interrupted exception during 'fast' crawls in workflow processor orbiter 2010-03-22 09:12:18 +00:00
  • ea69300857 fix bad floating navigators on little results lotus 2010-03-20 22:09:55 +00:00
  • bfb518cd47 some refactoring to get the LoaderDispatcher a little bit more independent from the switchboard orbiter 2010-03-20 10:28:03 +00:00
  • 36bd843ece for for RFC5322 comformance as suggested by Quix0r in http://forum.yacy-websuche.de/viewtopic.php?p=19585#p19585 orbiter 2010-03-20 10:23:47 +00:00
  • c855fc48c6 only load robots.txt for http and http protocol orbiter 2010-03-20 10:15:11 +00:00
  • 0465f28f7f applied 'null in rss2.js' fix from Quix0r, see http://forum.yacy-websuche.de/viewtopic.php?p=19612#p19612 orbiter 2010-03-20 09:58:05 +00:00
  • 748abfcffa added patches to prevent yacy-protocol DoS settings orbiter 2010-03-19 15:31:15 +00:00
  • e820ed061a avoiding excessive DNS lookups to determine localhost orbiter 2010-03-19 14:28:25 +00:00
  • 11983bc936 redesigned some parts of the parser entry point: - in all cases that the parser is entered it is a whole set of possible parsers computed according to given mime type and file extension, that means that all parsers are considered where the registered mime acceptance and extension acceptions matches. that may cause that several parsers are tried for the same file which will cause a success in cases where there was only the mime type was used to choose the right parser and the mime type was given wrongly by the host httpd. orbiter 2010-03-19 13:04:42 +00:00
  • de88200e11 - added Byte Order Mark recognition to serverObjects The BOM character FEFF may appear at the beginning of strings if some browsers append the characters %EF%BB%BF to input values. see http://en.wikipedia.org/wiki/Byte_order_mark orbiter 2010-03-19 10:58:40 +00:00
  • 89b4fff1c2 adopted ant script for new exif library orbiter 2010-03-12 12:36:38 +00:00
  • 24e5faee75 added exif parsing for jpg images orbiter 2010-03-12 12:23:38 +00:00
  • 82f76e1296 removed log line orbiter 2010-03-11 20:31:38 +00:00
  • 0f8004f9da enhanced html parser to recognize a href tags inside header tags orbiter 2010-03-11 17:52:07 +00:00
  • 3300930fc5 - (almost) fixed FTP crawler - integrated/fixed SMB crawler orbiter 2010-03-11 15:43:06 +00:00
  • 35d0057cb0 stopYACY.sh can now use curl orbiter 2010-03-11 00:12:53 +00:00
  • 61493a9a9f added more information about metadata in ViewFile.html orbiter 2010-03-11 00:11:14 +00:00
  • 1198b9989d bugfixes, more sorttable orbiter 2010-03-10 15:39:36 +00:00
  • 27b2998eb4 added searchtable function to more tables in interface you can now sort by any column in most tables in YaCy just by clicking on the headline column of the table orbiter 2010-03-10 10:05:41 +00:00
  • 9623d9e6d2 added a smb loader component for the YaCy crawler orbiter 2010-03-10 08:55:29 +00:00
  • c77fbd0390 added sorttable (http://www.kryogenix.org/code/browser/sorttable/) javascript library to make tables sortable orbiter 2010-03-09 23:40:16 +00:00
  • 3014e5f6f9 - integrated live search in the IndexControlURLs input window for URLs: this searchs for occurrences of the given word in URLs and presents them in a pop-up list below the input line - some bugfixes for the new robots table viewer orbiter 2010-03-09 15:44:11 +00:00
  • ae2f3f000f better handling of table copy abandon .. prevent memory leak orbiter 2010-03-09 13:32:15 +00:00
  • 0769517129 added a robots.txt monitor in the crawler monitor submenu orbiter 2010-03-09 11:31:15 +00:00
  • 48995e71c4 added soft-auth to general authentication scheme orbiter 2010-03-09 00:07:17 +00:00
  • 72f00dee59 removed never-used server access account function orbiter 2010-03-08 22:30:45 +00:00
  • 474bb4de82 ups orbiter 2010-03-07 23:32:18 +00:00
  • 8c88abf685 added follow-me link for twitter in status hints orbiter 2010-03-07 23:29:29 +00:00
  • 58d75a6bde allow more results for a single query at the same time if the client is not authorized. This is necessary for the search widget where the default number of results is now set to 20 instead of 10 to cause that a scroll bar is shown which is necessary to get a trigger for new searches for more results. orbiter 2010-03-07 22:49:20 +00:00
  • 57e1eae95e longer time-out for url fetching .. may help to show all that links that the statistic say for a search result orbiter 2010-03-07 22:23:08 +00:00
  • 9e639603e3 after frequent occurrences of 100% CPU usages and permanent blockings I try to disable a function in a method that may cause the problem when calling an external library (apache http client 3.x). The thread dump that shows the problem is attached here. orbiter 2010-03-07 21:19:23 +00:00
  • 4144927d94 show less errors orbiter 2010-03-07 21:02:08 +00:00
  • 736df39c9c Updated German translation de.lng: mainly ViewFile.html additions and removed (De)Select All from Table_API_p.html section mikeworks 2010-03-07 16:31:49 +00:00
  • b88f5fbb4b slightly changed crawling policy orbiter 2010-03-07 01:46:08 +00:00
  • de01fe0e6d fix for bug in url parser orbiter 2010-03-07 01:33:18 +00:00
  • 7684a575c4 fix for deletion of error database each time when YaCy starts up orbiter 2010-03-07 00:33:39 +00:00
  • f561e340c6 show more results of single domains when not authorized fully (up to 100) orbiter 2010-03-07 00:12:58 +00:00
  • c4bdb1e7f2 added one more option in ViewFile to show an iframe like for the orginal web page content but using the cache than the direct link to the content in the web. Upgraded the very old and previously not any more used CacheResource_p servlet to a new and working version. orbiter 2010-03-06 23:41:51 +00:00
  • c09a995930 better logging of double occurrences of urls in the crawler orbiter 2010-03-06 20:31:30 +00:00
  • 1bbe14d23f SVN 6716 unfortunately contained parts of the unfinished SMB integration. To fix compile errors the remaining parts of the SMB implementation stub is added with this commit. This adds the jcifs smb library. orbiter 2010-03-05 21:46:22 +00:00
  • 884b262130 - added a new Wiki Namespace Navigator - some redesign of Navigator data structures orbiter 2010-03-05 21:25:49 +00:00
  • b0c6d0108b fix for select-all toggle in tables servlet orbiter 2010-03-05 16:15:59 +00:00
  • 617dfbbd06 allo 'authorization by encoded password' also if requesting client is not from localhost but from the same host as yacy is running on. orbiter 2010-03-05 16:03:55 +00:00
  • 270fb38674 - fixed some bugs in Table viewer - added 'select all' feature in Tables_p - enhanced ViewFile.html: has now an input field to load arbitrary resources from the web and analyze them (!!!) - included the ViewFile servlet into the Index Administration menu - show in ViewFile if ressource is in url-db and/or in Web cache - bugfixes to BEncodedHeap and Tables management orbiter 2010-03-05 15:41:15 +00:00
  • 38d7a28cd2 fix in viewfile needed when ViewFile is called only with 'url' parameter orbiter 2010-03-05 12:24:15 +00:00
  • 599c3766c4 added authentication to automated API call orbiter 2010-03-04 14:10:03 +00:00
  • 727dd9b193 - fixed a bug in robots.txt parser - moved storage of robots.txt entries to WorkTables, so it is now possible to browse the robots entries with the table browser orbiter 2010-03-04 11:58:07 +00:00
  • 54af9e6b49 - added parsing of robots meta-tag in html headers to detect a noindexing request - added evaluation and indexing prevention in case that a noindexing is given in a html file orbiter 2010-03-03 23:32:56 +00:00
  • f336ed568d fir for parameters on reload lotus 2010-03-03 21:14:12 +00:00
  • ddc21e7b73 adapted banner colors to current standard template lotus 2010-03-03 21:10:02 +00:00
  • e76f1e6cc0 Update property name bookmarks to display_bookmarks (changed in SVN 5554) to fix the broken Bookmark RSS feed - Fixes RSS part of http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1824 mikeworks 2010-02-28 11:22:01 +00:00
  • c52dec0c06 Updated German translation de.lng: Translated new entries in Network.html and Status_p.inc mikeworks 2010-02-27 03:44:25 +00:00
  • cc074c1a36 Renamed, removed and added license information for each jar archive in external lib folder mikeworks 2010-02-27 03:04:11 +00:00
  • 8a19be24de tell non-windows users about the tray-icon lotus 2010-02-26 19:44:23 +00:00
  • f5ec7ad077 replaced four old libraries with latest version orbiter 2010-02-26 14:14:50 +00:00
  • 6b89f681c5 added bing, dbpedia and wolfram alpha to the compare-search options orbiter 2010-02-26 10:34:36 +00:00
  • 475ffabfa1 Added License (Apache 1.1) information for Jakarta ORO library 2.0.7 (2.0.8 available) mikeworks 2010-02-26 04:26:04 +00:00
  • 46c4f8b68a better look-ahead into the crawl queue: show more on crawl monitor orbiter 2010-02-24 23:11:58 +00:00
  • 4b6efe3b48 more ergonomic default values for crawl start orbiter 2010-02-24 22:33:51 +00:00
  • 7b546415dc added svn6695 for windows lotus 2010-02-24 14:58:53 +00:00
  • f175f9a2d3 changed way how number of search requests are counted: so far only search requests at the remote search interface had been counted. This was done to protect the privacy of searchers, because counting was not done and published at the own search interface. This caused that no search requests of robinson peers had been counted, becuase they cannot be counted at remote peer. This change introduces a distinction of locally done search requests at the local search interface from search requests that are on the local interface but had been submitted from a remote IP without authentication. Now 3 counters are maintained: - partial count of remote searches - total count of local searches on robinson peers from non-authenticated clients - total count of local searches on robinson peers from localhost or authenticated clients In the global statistic of search requests now the first two counters of the three cases are added Because we habe a large number of robinson peers with a large number of remote non-authenticated requests the statistic should show at least three times of the number of search requests. orbiter 2010-02-24 13:53:55 +00:00
  • 84222e3b4f fix for auto-updater: delete old libraries before copy of new one orbiter 2010-02-24 13:46:50 +00:00
  • cd6de83905 next try for for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2703 (reverted 6692) sixcooler 2010-02-23 15:59:58 +00:00
  • bfe4693e9a fix for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2703 sixcooler 2010-02-23 13:46:56 +00:00