Mirrors/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2025-11-06 23:56:13 -05:00
Code Issues Packages Projects Releases Wiki Activity

Commit Graph

  • 621b473b18 * removed some warnings of findbugs (http://findbugs.sf.net) - removed unnecessary code (unused variables, String.toString) - corrected some calculations (cast int to double or long ;) - improved little performance (using Integer.valueOf() instead of new Integer) - log if some File-actions fail (mkdir(), delete(), ...) and some ignored exceptions - finalized some (more) fields - finally close some streams - made inner classes static if not using environment - generalized some equals (from specificClass to Object) - fixed some potential nullpointer accesses danielr 2008-08-06 19:43:12 +00:00
  • 5627ca1c60 * rename yacy.nsi to build.nsi lotus 2008-08-06 19:32:55 +00:00
  • d72846f898 * moved nsisant library in seperate libbuild-directory to keep root directory clean f1ori 2008-08-06 17:02:56 +00:00
  • 3f871b57de - again an error in defaults/autoReCrawl.conf - you have to delte or edit DATA/SETTINGS/autoReCrawl.conf to fix this problem, sorry! apfelmaennchen 2008-08-05 18:57:25 +00:00
  • 0500b1179e added a 2 min start up delay to serverBusyThread autoReCrawl to avoid a Null Pointer Exception... apfelmaennchen 2008-08-05 05:50:15 +00:00
  • 3b6229d2f3 small fix - parseBoolean doesn't recognize 1 and 0 as true and false in autoReCrawl.conf apfelmaennchen 2008-08-04 21:57:57 +00:00
  • e1574fe02e - added autoReCrawl folders to bookmarks (DATA/SETTINGS/autoReCrawl.conf) - the serverBusyThread checks folders every 60 min. (==> autoReCrawl_idlesleep in yacy.conf) - added option to create bookmarks from CrawlStart URL apfelmaennchen 2008-08-04 20:43:36 +00:00
  • ebb40d324b enhanced memory chart: shows now also the size of the word cache as third vector. The PPM is now shown without a scale, but with a new anotation at the chart entry. orbiter 2008-08-04 10:47:26 +00:00
  • 17b7845eb5 * refactoring - moved constants from plasmaSwitchboard to own class (all 232 ;) - moved remoteProxy-Methods to httpRemoteProxyConfig, better names - removed some unnecessary code (else-statements) * formatting (correct indentation) * minor bugfixes (due to findbugs.sf.net) * hopefully fixed "missing quote" (announcing StringParts as UTF-8) danielr 2008-08-02 13:57:00 +00:00
  • 3bb870bfcd added final where possible danielr 2008-08-02 12:12:04 +00:00
  • 7e92484400 fix for open browser on windows 2000 lotus 2008-08-01 12:58:36 +00:00
  • b0724e5ec0 * add config option to disable cookie monitoring (disabled by default) f1ori 2008-07-30 21:19:06 +00:00
  • f0dfb858d8 reverted balloon messages from r5014. don't try to annoy the user. ;) lotus 2008-07-30 07:42:02 +00:00
  • cafa1a952f - performance settings refactoring: made headlines to fieldsets - show performance profiles more prominent for easier configuration lotus 2008-07-30 07:37:11 +00:00
  • 6952b13c08 * fix automatic decrementing accepted remote crawl limit: -> http://forum.yacy-websuche.de/viewtopic.php?f=5&t=1355 f1ori 2008-07-29 18:52:48 +00:00
  • d02a26d642 * added lock icon to ConfigAppearance menu item f1ori 2008-07-28 21:09:19 +00:00
  • 0b2f67577e Index Transfer: - fix for chunk size calculation - fix: if chunk size was 1, an infinite selection loop ran because no entries were found. if chunk size fails <=3 it will be set back to 500 lotus 2008-07-27 18:53:51 +00:00
  • 0baa8d651e * marked some text as labels for checkboxes f1ori 2008-07-27 11:39:42 +00:00
  • 694084c570 fix for NPE on shutdown lotus 2008-07-27 06:59:56 +00:00
  • f9fb2d0fee hide memory tables by default lotus 2008-07-27 06:59:24 +00:00
  • 5f77f55ed7 possible fix for negative speed values lotus 2008-07-27 06:58:35 +00:00
  • 79dba5445d init-script: fixed some variables to general defaults danielr 2008-07-24 19:04:35 +00:00
  • 785688ef82 - creating pid-file under linux - added modified init-script (using startYACY.sh instead of doing the same thing) danielr 2008-07-24 18:56:29 +00:00
  • 50ef5c406f - refactoring of robots parser (removed opaque Objects[] result vector) - added Allow-component to robots result object orbiter 2008-07-24 11:54:37 +00:00
  • 7913bdb75b Flextable: filename in errormessage if inconsistent danielr 2008-07-24 08:22:36 +00:00
  • d42eae25f8 yacyTray: fix for unproper shutdown some messages lotus 2008-07-24 06:49:30 +00:00
  • 7cec908813 dont't show HTTPC debug messages danielr 2008-07-23 17:08:40 +00:00
  • 5f41913043 fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1347&hilit=&p=9216#p9216 orbiter 2008-07-20 21:52:51 +00:00
  • c3d461d191 - removed superfluous copyright statement - updated my email address orbiter 2008-07-20 17:14:51 +00:00
  • 3ca98fee42 removed superfluous copyright statement orbiter 2008-07-20 00:21:07 +00:00
  • c049d80fbd fixed login problem with yacy as proxy (POST and Cookies) danielr 2008-07-19 15:10:00 +00:00
  • 62afea0c9f some improvements for yacyTray lotus 2008-07-18 14:17:52 +00:00
  • af616a92ff fixed welcome message above image on custom search page danielr 2008-07-18 13:41:06 +00:00
  • 7c110e07f0 removed debug danielr 2008-07-16 12:39:38 +00:00
  • 10d01fdd0d addendum to 5002 danielr 2008-07-16 09:47:48 +00:00
  • eadc204130 gzip POST wiederholbar gemacht (macht transferURL stabiler) danielr 2008-07-16 09:46:25 +00:00
  • d14e8d348f - times of LOCAL_SEARCH are shown in milliseconds (also in yacysearch.java ;) danielr 2008-07-15 17:35:02 +00:00
  • 28c39e2aa4 fix for new starter files lotus 2008-07-14 18:41:11 +00:00
  • b63a519456 fixed build problem danielr 2008-07-14 17:44:49 +00:00
  • e12438142e - warning instead of NPE if urlHash is not in index of CrawResults - times of LOCAL_SEARCH are shown in milliseconds danielr 2008-07-14 09:12:58 +00:00
  • 5784070f6e added user agent lulabad 2008-07-14 08:38:11 +00:00
  • 3e26aca356 added startYACY_debug.bat to dist Target lulabad 2008-07-13 20:32:28 +00:00
  • fa695c2d9f tray is now only shown on Windows and doesn't block on linux lotus 2008-07-13 19:03:38 +00:00
  • d77ed28e2f temporary disabled tray because of flaws on only-shell-linux lotus 2008-07-13 08:41:39 +00:00
  • adeb881288 fix for building Windows installer lotus 2008-07-13 08:17:20 +00:00
  • b704cf1fbc * no need to update the searchpage links anymore lotus 2008-07-13 07:55:20 +00:00
  • 0bb7c42bc8 rename of Windows start scripts * startYACY.bat -> startYACY_debug.bat * startYACY_noconsole.bat -> startYACY.bat lotus 2008-07-13 07:54:51 +00:00
  • f8a1e3175e new yacyTray this will make a YaCy icon in the tray area on supported platforms enabled by default the search page will open on double click lotus 2008-07-13 07:51:45 +00:00
  • 4a7e5f377e added translation for new customization pages daburna 2008-07-12 12:27:33 +00:00
  • 05c26d58d9 fixed missing remove operation in balancer orbiter 2008-07-11 12:03:18 +00:00
  • 606b323a2d fixed bug that appeared when a new crawl ist started orbiter 2008-07-11 10:59:06 +00:00
  • 38eb5bd1ee fixed a bug in kelondroBLOBHeap. The following files are probably inconsistent and should be deleted: DATA/HTCACHE/responseHeader.heap DATA/PLASMADB/crawlRobotsTxt.heap orbiter 2008-07-11 10:38:20 +00:00
  • 28d5703f8a - fixed a bug in Robots.txt loader which could have caused that robots.txt files had been loaded from the same domain more than once - patch in BLOBHeap to prevent OOM during startup orbiter 2008-07-11 09:12:54 +00:00
  • 7b1c9e6aee discovered and removed a (possibly large) memory leak: many classes used the kelondroMapDataMining (was: kelondroMapObjects) which adds statistical functions to the kelondroMap (was: kelondroObjects), but these functions were not used by these classes. Especially the HTCACHE and robots.txt database allocate a very large number of objects for statistical use, but never used them. By replacing the kelondroMapDataMining with the kelondroMap object for these classes now less memory is allocated. orbiter 2008-07-11 07:34:48 +00:00
  • 0f5fe8cc53 refactoring of method calling for objects from kelondroMapDataMining orbiter 2008-07-11 07:15:46 +00:00
  • 01d1ae6676 patch for negative time in case that the time of the computer is changed orbiter 2008-07-11 07:05:08 +00:00
  • bac758bc5a * update to JRE 6u7 lotus 2008-07-11 05:26:55 +00:00
  • 4acf0a61cd refactoring of kelondroObjects (mainly renaming to kelondroMap) orbiter 2008-07-10 22:08:16 +00:00
  • 441e9c861e fix for npe in HTCache cleaning process orbiter 2008-07-10 21:30:39 +00:00
  • f7aaeb3fad created new main menu entry 'Customization and Integration' - moved some already existing servlets to this menu - renamed the skin servlet to appearance - added a set-to-default-button to the search page appearance setting - removed the peer profile servlet which is now replaced by a field in the new appearance servlet orbiter 2008-07-10 19:57:09 +00:00
  • 5488543b8f disabled disk usage logpoints lotus 2008-07-10 07:30:50 +00:00
  • 1e6d12f146 Major update to BLOB data structures: - introduced a new BLOB file format: kelondroBLOBHeap. This is a flat file with an index in RAM. very similar to the eco-tables, but with flexible value sizes. It will replace the kelondroBLOBTree, which is based on a kelondroTree, a file-AVL-based index data structure. - the HTCACHE header file was replaced by the new blob heap file structure - the robots.txt file was replaced by the new blob heap file structure - the robots parser was enhanced (bugfixing for double-loading of the same robots.txt) - other BLOB-dependent data structures were prepared to use also the new BLOB heap - fixed a bug in the snippet fetch process: the file header was not written to the header index There should now be less IO during snippet fetch and during crawling orbiter 2008-07-10 00:47:37 +00:00
  • 81f75f5056 - removed unnecessary classes (these objects are much easier to handle using generics) - generalized BLOB referencing. This is the preparation to use another BLOB class, the kelondroHeap orbiter 2008-07-07 23:52:53 +00:00
  • b38f467e3c better SRU compliance orbiter 2008-07-07 21:50:24 +00:00
  • 3980dd7554 * maske ampersan & Co. in xbel export f1ori 2008-07-07 21:30:11 +00:00
  • 7052f2f61f - added copyright header of ResourceObserver - commented/removed some code to eliminate code warnings orbiter 2008-07-07 00:40:45 +00:00
  • 1400cdc91e - refactoring of resourceObserver (moved it to crawler) - partly redesign of diskUsage: little bit more functional behavior, less side effects, better error case handling - the resourceObserver can now show a error message if the diskUsage is 'out of order' orbiter 2008-07-07 00:03:37 +00:00
  • f0012554de * improved localsearch.sh: * read port from DATA/SETTINGS/yacy.conf * use curl or wget, depending on what is available f1ori 2008-07-06 16:48:41 +00:00
  • b6301a54fa * added class ListDirs to provoid generic listing of directories in systemdirectories and jar-files * yacy runs, when classes are in a jar-file (->build-jar ant-target) f1ori 2008-07-06 14:11:40 +00:00
  • f2e2d09916 - fix for index transfer - imported a random startpoint function from plasmaDHTChunk in case there was already a gap at the beginning of the index, the transfer process was endless selecting from first startpoint tested & working on my index lotus 2008-07-06 13:16:17 +00:00
  • a6719dfd2b - refactoring of robots parser - no more keep-order parameter in remove (it was not possible to make this strict, and not useful) - some small enhancements in balancer - robots parser without references in switchboard - changes synchronization in robots orbiter 2008-07-05 00:35:20 +00:00
  • e81be7d4f2 added many missing user-agent declarations for yacy http client connections. the most important fix was the addition of the yacybot user-agent for robots.txt loading, because web masters look for that access to see if the crawler behaves correctly. orbiter 2008-07-04 11:03:03 +00:00
  • 474e29ce4a added options to configure the 'corporate identity'-icons, the home page link and the greeting line from the skin menue. Additionally an example is given there how to integrate a search page with an iframe. Please see the skin menu. orbiter 2008-07-03 23:37:04 +00:00
  • 474659a71f - modified and enhanced the crawl balancer: better list export, fixing of damaged crawl queue at start-up, re-sorting at start-up to enhance domain order - added option to set minimum crawl delta for domains in balancer - added default values to crawl deltas in yacy.init - added configuration for these deltas in performance queues - enhanced performance setting computation (more time for indexing queue for a faster flush - remote crawling is now enabled during local crawling if indexer has space and time for more links - added database stub for new distributed file system - refactoring of time computation to get an abstraction level that will be used by a TTL rule in new distributed file system orbiter 2008-07-03 13:08:37 +00:00
  • 1afc36d9a8 * update to JRE 6u6 * fix for proper translation lotus 2008-07-02 13:36:42 +00:00
  • 8e179f6588 removed option to do a re-crawl with a period of minutes. Such a short time does not make sense and it may cause endless indexing loops. The removing of the option will ensure that a misuse is prevented. orbiter 2008-07-01 23:47:33 +00:00
  • 080cda97ef added another peer selection rule: - select also non-robinson (dht-) peers if their peer tags match with search words - the peer tag '*' can now act as catch-all rule: shall be selected always orbiter 2008-06-30 23:04:32 +00:00
  • d37fd064f9 changed peer selection for search targets: - less dht targets are selected - more other peers are selected: all robinson peers with more than one million urls orbiter 2008-06-30 22:42:52 +00:00
  • 69aac0d74c modified the diskUsage class regarding the following two aspects: 1. The usage and dependency of the plasmaSwitchboad was used many times in the past but this was a bad mistake. The classes should be independent from the switchboard to support a better abstraction. Therefore the object was removed. The parameters from the switchboard are computed outside and then handed over. 2. the class is considered as a tightly connected to hardware resources. Classes which handle data that cannot be replicated because it would need to replicate hadware should not support dynamic object allocation, but should be coded as collection of private static methods. Therefore all class objects had been transformed into static private objects. orbiter 2008-06-30 21:47:53 +00:00
  • da917cf4b1 undo reduced menu danielr 2008-06-30 07:11:13 +00:00
  • 0c1dc703e4 - set staticIP at startUp - added setting for reduced menu (simpleMenu) danielr 2008-06-29 18:35:15 +00:00
  • f6b92456c0 confirm bookmark delete lotus 2008-06-27 05:48:29 +00:00
  • f7f9ceb967 diskUsage: replaced blocking sleep with semaphore danielr 2008-06-26 12:05:12 +00:00
  • 4a53649ee7 fixed dht-urls and ranking distribution log statistics * NOTE: please have in mind that there can be whitespaces in pathnames lotus 2008-06-26 07:12:03 +00:00
  • 8d83185cb4 fixed dht-chunks/protocol log statistics lotus 2008-06-25 08:15:07 +00:00
  • 63eadfdf84 fixed unlimited FileSizeLimit danielr 2008-06-24 19:11:27 +00:00
  • 2dc7c00c1c fixed indexing log statistics lotus 2008-06-24 07:01:04 +00:00
  • dba7ba079e fixed NPE seen with queues_p.xml (serverClassLoader finds already loaded classes) danielr 2008-06-23 16:55:46 +00:00
  • c5e5c554d2 corrected unresolved dings daburna 2008-06-23 09:43:26 +00:00
  • 273fb01142 revert last fix; was wrong det 2008-06-21 21:07:28 +00:00
  • b6f50851fa fix memory requirement calculation det 2008-06-21 20:58:57 +00:00
  • ac85c52bae better readability for MIN_FREE_DISK_SPACE lotus 2008-06-21 10:20:36 +00:00
  • 54a73b58cf fixed restart on Windows when directory had spaces in it's name lotus 2008-06-21 09:19:26 +00:00
  • f30f976f2d installer downloads Java to shared documents and creates a shortcut on the desktop in case user is no admin lotus 2008-06-21 08:42:06 +00:00
  • 0a956d1b11 *) removes focus on text field on search result page (http://forum.yacy-websuche.de/viewtopic.php?f=9&t=1257) low012 2008-06-20 16:46:30 +00:00
  • 2a53551423 removed unused code from installer lotus 2008-06-20 14:22:26 +00:00
  • 609aaf0df3 rework of the windows part det 2008-06-20 12:13:06 +00:00
  • 1a4f26ba30 exclude HTDOCS from recursiv scan det 2008-06-20 10:03:49 +00:00
  • 6c07e894d9 add needed sleep det 2008-06-20 09:53:23 +00:00
  • d742cc080c Fix for RAMCache not flushing see: http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1255 hermens 2008-06-19 18:27:48 +00:00