Mirrors/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2025-07-23 09:24:39 -04:00
Code Issues Packages Projects Releases Wiki Activity

Commit Graph

  • 01d1ae6676 patch for negative time in case that the time of the computer is changed orbiter 2008-07-11 07:05:08 +00:00
  • bac758bc5a * update to JRE 6u7 lotus 2008-07-11 05:26:55 +00:00
  • 4acf0a61cd refactoring of kelondroObjects (mainly renaming to kelondroMap) orbiter 2008-07-10 22:08:16 +00:00
  • 441e9c861e fix for npe in HTCache cleaning process orbiter 2008-07-10 21:30:39 +00:00
  • f7aaeb3fad created new main menu entry 'Customization and Integration' - moved some already existing servlets to this menu - renamed the skin servlet to appearance - added a set-to-default-button to the search page appearance setting - removed the peer profile servlet which is now replaced by a field in the new appearance servlet orbiter 2008-07-10 19:57:09 +00:00
  • 5488543b8f disabled disk usage logpoints lotus 2008-07-10 07:30:50 +00:00
  • 1e6d12f146 Major update to BLOB data structures: - introduced a new BLOB file format: kelondroBLOBHeap. This is a flat file with an index in RAM. very similar to the eco-tables, but with flexible value sizes. It will replace the kelondroBLOBTree, which is based on a kelondroTree, a file-AVL-based index data structure. - the HTCACHE header file was replaced by the new blob heap file structure - the robots.txt file was replaced by the new blob heap file structure - the robots parser was enhanced (bugfixing for double-loading of the same robots.txt) - other BLOB-dependent data structures were prepared to use also the new BLOB heap - fixed a bug in the snippet fetch process: the file header was not written to the header index There should now be less IO during snippet fetch and during crawling orbiter 2008-07-10 00:47:37 +00:00
  • 81f75f5056 - removed unnecessary classes (these objects are much easier to handle using generics) - generalized BLOB referencing. This is the preparation to use another BLOB class, the kelondroHeap orbiter 2008-07-07 23:52:53 +00:00
  • b38f467e3c better SRU compliance orbiter 2008-07-07 21:50:24 +00:00
  • 3980dd7554 * maske ampersan & Co. in xbel export f1ori 2008-07-07 21:30:11 +00:00
  • 7052f2f61f - added copyright header of ResourceObserver - commented/removed some code to eliminate code warnings orbiter 2008-07-07 00:40:45 +00:00
  • 1400cdc91e - refactoring of resourceObserver (moved it to crawler) - partly redesign of diskUsage: little bit more functional behavior, less side effects, better error case handling - the resourceObserver can now show a error message if the diskUsage is 'out of order' orbiter 2008-07-07 00:03:37 +00:00
  • f0012554de * improved localsearch.sh: * read port from DATA/SETTINGS/yacy.conf * use curl or wget, depending on what is available f1ori 2008-07-06 16:48:41 +00:00
  • b6301a54fa * added class ListDirs to provoid generic listing of directories in systemdirectories and jar-files * yacy runs, when classes are in a jar-file (->build-jar ant-target) f1ori 2008-07-06 14:11:40 +00:00
  • f2e2d09916 - fix for index transfer - imported a random startpoint function from plasmaDHTChunk in case there was already a gap at the beginning of the index, the transfer process was endless selecting from first startpoint tested & working on my index lotus 2008-07-06 13:16:17 +00:00
  • a6719dfd2b - refactoring of robots parser - no more keep-order parameter in remove (it was not possible to make this strict, and not useful) - some small enhancements in balancer - robots parser without references in switchboard - changes synchronization in robots orbiter 2008-07-05 00:35:20 +00:00
  • e81be7d4f2 added many missing user-agent declarations for yacy http client connections. the most important fix was the addition of the yacybot user-agent for robots.txt loading, because web masters look for that access to see if the crawler behaves correctly. orbiter 2008-07-04 11:03:03 +00:00
  • 474e29ce4a added options to configure the 'corporate identity'-icons, the home page link and the greeting line from the skin menue. Additionally an example is given there how to integrate a search page with an iframe. Please see the skin menu. orbiter 2008-07-03 23:37:04 +00:00
  • 474659a71f - modified and enhanced the crawl balancer: better list export, fixing of damaged crawl queue at start-up, re-sorting at start-up to enhance domain order - added option to set minimum crawl delta for domains in balancer - added default values to crawl deltas in yacy.init - added configuration for these deltas in performance queues - enhanced performance setting computation (more time for indexing queue for a faster flush - remote crawling is now enabled during local crawling if indexer has space and time for more links - added database stub for new distributed file system - refactoring of time computation to get an abstraction level that will be used by a TTL rule in new distributed file system orbiter 2008-07-03 13:08:37 +00:00
  • 1afc36d9a8 * update to JRE 6u6 * fix for proper translation lotus 2008-07-02 13:36:42 +00:00
  • 8e179f6588 removed option to do a re-crawl with a period of minutes. Such a short time does not make sense and it may cause endless indexing loops. The removing of the option will ensure that a misuse is prevented. orbiter 2008-07-01 23:47:33 +00:00
  • 080cda97ef added another peer selection rule: - select also non-robinson (dht-) peers if their peer tags match with search words - the peer tag '*' can now act as catch-all rule: shall be selected always orbiter 2008-06-30 23:04:32 +00:00
  • d37fd064f9 changed peer selection for search targets: - less dht targets are selected - more other peers are selected: all robinson peers with more than one million urls orbiter 2008-06-30 22:42:52 +00:00
  • 69aac0d74c modified the diskUsage class regarding the following two aspects: 1. The usage and dependency of the plasmaSwitchboad was used many times in the past but this was a bad mistake. The classes should be independent from the switchboard to support a better abstraction. Therefore the object was removed. The parameters from the switchboard are computed outside and then handed over. 2. the class is considered as a tightly connected to hardware resources. Classes which handle data that cannot be replicated because it would need to replicate hadware should not support dynamic object allocation, but should be coded as collection of private static methods. Therefore all class objects had been transformed into static private objects. orbiter 2008-06-30 21:47:53 +00:00
  • da917cf4b1 undo reduced menu danielr 2008-06-30 07:11:13 +00:00
  • 0c1dc703e4 - set staticIP at startUp - added setting for reduced menu (simpleMenu) danielr 2008-06-29 18:35:15 +00:00
  • f6b92456c0 confirm bookmark delete lotus 2008-06-27 05:48:29 +00:00
  • f7f9ceb967 diskUsage: replaced blocking sleep with semaphore danielr 2008-06-26 12:05:12 +00:00
  • 4a53649ee7 fixed dht-urls and ranking distribution log statistics * NOTE: please have in mind that there can be whitespaces in pathnames lotus 2008-06-26 07:12:03 +00:00
  • 8d83185cb4 fixed dht-chunks/protocol log statistics lotus 2008-06-25 08:15:07 +00:00
  • 63eadfdf84 fixed unlimited FileSizeLimit danielr 2008-06-24 19:11:27 +00:00
  • 2dc7c00c1c fixed indexing log statistics lotus 2008-06-24 07:01:04 +00:00
  • dba7ba079e fixed NPE seen with queues_p.xml (serverClassLoader finds already loaded classes) danielr 2008-06-23 16:55:46 +00:00
  • c5e5c554d2 corrected unresolved dings daburna 2008-06-23 09:43:26 +00:00
  • 273fb01142 revert last fix; was wrong det 2008-06-21 21:07:28 +00:00
  • b6f50851fa fix memory requirement calculation det 2008-06-21 20:58:57 +00:00
  • ac85c52bae better readability for MIN_FREE_DISK_SPACE lotus 2008-06-21 10:20:36 +00:00
  • 54a73b58cf fixed restart on Windows when directory had spaces in it's name lotus 2008-06-21 09:19:26 +00:00
  • f30f976f2d installer downloads Java to shared documents and creates a shortcut on the desktop in case user is no admin lotus 2008-06-21 08:42:06 +00:00
  • 0a956d1b11 *) removes focus on text field on search result page (http://forum.yacy-websuche.de/viewtopic.php?f=9&t=1257) low012 2008-06-20 16:46:30 +00:00
  • 2a53551423 removed unused code from installer lotus 2008-06-20 14:22:26 +00:00
  • 609aaf0df3 rework of the windows part det 2008-06-20 12:13:06 +00:00
  • 1a4f26ba30 exclude HTDOCS from recursiv scan det 2008-06-20 10:03:49 +00:00
  • 6c07e894d9 add needed sleep det 2008-06-20 09:53:23 +00:00
  • d742cc080c Fix for RAMCache not flushing see: http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1255 hermens 2008-06-19 18:27:48 +00:00
  • 6b7e873962 resourceObserver refactoring and some synchronisation for console output danielr 2008-06-19 12:40:44 +00:00
  • 01e9c7320e PerformanceProfile: don't change peerPing, peerSeed and cleanUp danielr 2008-06-19 12:39:45 +00:00
  • 6bdd99e065 - more asserts to solve the ooB-problem - better caching (?), lets see how it behaves orbiter 2008-06-18 21:08:56 +00:00
  • 24fb2d0922 distribute searchtest lotus 2008-06-18 14:08:44 +00:00
  • b928ae492a some code-cleanup and possible speed enhancements in different core methods orbiter 2008-06-17 23:56:39 +00:00
  • 6a9cc29cdd workaround for IndexOutOfBoundsException in ResultURLs.getExecutorHash() seen @ CrawlResults.html?process=4 danielr 2008-06-17 18:56:04 +00:00
  • c998dc6556 - added security functions to flush url and search caches in case that memory is full orbiter 2008-06-16 21:39:58 +00:00
  • f4ae8082c3 - better error analysis for ooRange Exception in kelondroBase64Ordering - quadcore support for kelondroRowSet array ordering orbiter 2008-06-15 23:25:57 +00:00
  • f8af00d7e8 added some experimental jvm startup properties which helped the metager peer to get more stable. orbiter 2008-06-15 21:41:46 +00:00
  • 84cbe75005 more asserts orbiter 2008-06-15 00:04:59 +00:00
  • e269c12710 small changes in partition routine orbiter 2008-06-14 23:17:56 +00:00
  • 31efb8fbee - fix for LOG path generation when the DATA/LOG does not exists (fix for bug introduced in SVN 4923) - some more/better asserts - slight performance enhancements in remove method in index management. Works for all who do not run using asserts (the majority) orbiter 2008-06-14 22:51:47 +00:00
  • 877299cc74 better installer on Windows Vista lotus 2008-06-14 18:34:12 +00:00
  • 726218dd4a fixed logging danielr 2008-06-14 14:10:53 +00:00
  • 21c87c36e3 added a log line orbiter 2008-06-14 11:57:46 +00:00
  • 5ea4fcd946 eindeutschung der Performance-Profile danielr 2008-06-14 11:02:36 +00:00
  • 68c38c2d34 - WatchCrawler shows status without JavaScript - Performance can be scaled + DHT-profile - names for pool-threads - some small refactorings danielr 2008-06-14 10:24:58 +00:00
  • da4b1b5c0d added a little translation for the new searchinterface daburna 2008-06-13 11:01:19 +00:00
  • d4fa7834f7 added flori and me daburna 2008-06-13 10:05:31 +00:00
  • fc79f013c4 better solution to update shortcut lotus 2008-06-12 20:04:32 +00:00
  • c0dfe49743 also exclude collection.0028.commons and RANKING at startup check det 2008-06-12 15:17:01 +00:00
  • cc40cde9e7 update of readme: use Java 1.5 instead of 1.4.2 orbiter 2008-06-12 15:09:19 +00:00
  • 11656741f1 exclude LOCALE and RELEASE at startup check det 2008-06-12 11:25:25 +00:00
  • 994c609cf8 added new shell script to do a web search from the terminal orbiter 2008-06-11 21:33:36 +00:00
  • 48edbef5c7 * fix: display proper port on 1st startup * new message on portchange * first implementation of external link-update for search page (still inactive) lotus 2008-06-11 19:04:39 +00:00
  • 0727bb1e63 rework of console message handling; add of debugging output det 2008-06-11 18:43:12 +00:00
  • e9dd0e6831 added a test script for profiling of local search speed and a list of search words current result: 120 milliseconds for a single search during a sequence of 100 searches, while the crawler is running with 500 PPM! orbiter 2008-06-11 10:36:37 +00:00
  • 43c47218ef fox for open browser on Windows lotus 2008-06-11 10:18:01 +00:00
  • f5ef7f222e - fixed a bug in parser (directory paths had not been recognized) - no access check when a search is made only local without snippet fetch - added comment and status message in resourceObserver (this takes very long at startup time!) orbiter 2008-06-11 09:54:58 +00:00
  • 48f87db7f1 added performance profiles (to be fine-tuned) danielr 2008-06-10 19:23:38 +00:00
  • ed24eab737 small fix for windows in resource observer lotus 2008-06-10 19:19:54 +00:00
  • 73051ecb37 corrected wrong link daburna 2008-06-09 14:30:16 +00:00
  • 0c07dd5e2f JRE 1.5 is also good lotus 2008-06-09 07:22:00 +00:00
  • d281d62d8a windows installer checks for JRE lotus 2008-06-09 07:14:21 +00:00
  • 5e69ee67f9 including NOTICE lotus 2008-06-08 16:36:28 +00:00
  • 89cdf5105f * removed axis from libx * don't install libs on linux with ant-target (libs should be in seperate packages) f1ori 2008-06-08 11:13:56 +00:00
  • 75b4a5ced4 reinstate old timeout values for transferRWI and transferURL hermens 2008-06-07 23:48:46 +00:00
  • 3330181aa0 refactoring: find a better way to store BLOBs; generalize current BLOG data structure (kelondroDyn) and prepare it to replace it with something better. The best candidate is the kelondroHeap, which will become the kelondroBLOBHeap; removed also some never-used classes orbiter 2008-06-07 23:12:24 +00:00
  • 9a9737a54e fix for "no more elements available" exception orbiter 2008-06-07 22:07:25 +00:00
  • 9e7c60f227 rename for last commit lotus 2008-06-07 18:40:58 +00:00
  • 0ce1f951ed * fix: files without extension are now included in dist-target * fix: windows-installer works again lotus 2008-06-07 18:40:20 +00:00
  • af8f7b8bba add image rising while mouse over in Collage det 2008-06-07 18:33:04 +00:00
  • 260553c3a5 better messages lotus 2008-06-07 14:24:11 +00:00
  • 6afeb535cd another bugfix for the windows drive check det 2008-06-07 12:51:07 +00:00
  • b416af7568 bugfix for the windows drive check det 2008-06-06 20:38:09 +00:00
  • 4b71912e76 fixed wrong class name danielr 2008-06-06 17:13:31 +00:00
  • 7feae906aa - organize imports - removed potential null pointer accesses - removed unnecessary casts danielr 2008-06-06 16:01:27 +00:00
  • f597185026 Initial import of the resource observer framework det 2008-06-06 13:10:21 +00:00
  • 40e1b989ea new release cycle orbiter 2008-06-05 21:30:22 +00:00
  • c10eaf9bdb - fix for pop-up page upon first start - added comments in opensearchdescription to explain fast mode - release 0.59 orbiter 2008-06-05 20:46:31 +00:00
  • d3037c2950 Accept all SSL-certificates (not only valid and self-signed), but put a warning into log file danielr 2008-06-05 15:21:43 +00:00
  • 1b906053ad -updated german language file daburna 2008-06-05 15:14:27 +00:00
  • 0e1c7dfaaf small fix for testing environment that should not affect a production environment orbiter 2008-06-05 13:08:40 +00:00
  • e0e7f86f82 some bugfixes for the peer-ping process orbiter 2008-06-05 12:52:27 +00:00
  • cbe722c480 small code cleanUp danielr 2008-06-05 12:41:55 +00:00