Mirrors/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2025-06-20 03:56:07 -04:00
Code Issues Packages Projects Releases Wiki Activity

Commit Graph

  • b7f4a1521b added options to switch on or off the kelondroFlexTable for NURL, EURL and PreNURL orbiter 2006-08-24 22:21:22 +00:00
  • c26da4893b turned back NURL usage of kelondroTree, kelondroFlexTable has still problems with deleted entries orbiter 2006-08-24 10:03:38 +00:00
  • db1eae0227 * simplified initialization of database objects * replaced kelondroTree for NURLs by kelondroFlex * replaced kelondroTree for EURLs by kelondroFlex take care, may be very buggy please finish crawls before updating. crawls will be lost. orbiter 2006-08-24 02:19:25 +00:00
  • 0b73f2b132 Repair DNS prefetch during cacheScan hermens 2006-08-24 01:31:08 +00:00
  • e34e07e0a1 - Changed back to dev namescheme and new 0.461 - Corrected some errors in News.html rramthun 2006-08-23 19:39:48 +00:00
  • 2d9478d203 installer for 0.46 allo 2006-08-23 13:09:48 +00:00
  • 27a159b401 * documentation update * removed doc from release * release information in doc/News.html * release 0.46 orbiter 2006-08-23 11:36:09 +00:00
  • f80f776b89 *) Trying to solve NullpointerException problem in function addURLtoErrorDB See: http://www.yacy-forum.de/viewtopic.php?t=2705 theli 2006-08-23 10:23:20 +00:00
  • d78b824e85 fixed problem with default path after first start-up orbiter 2006-08-22 13:35:51 +00:00
  • 57d50df858 documentation update orbiter 2006-08-22 01:46:17 +00:00
  • c2264962d1 *) Fixed a bug. low012 2006-08-21 21:34:45 +00:00
  • b387bd47db *) Added a Wiki Code help page and links to it at every textbox Wiki Code can be used in. Maybe this will encourage people to play around with the Wiki and the Blog more. low012 2006-08-21 21:23:29 +00:00
  • 1c99b5a484 *)fixed logging for urldbcleanup *)changed exception handling in urldbcleanup so that it shows NullPointerException correctly *)added more Blacklisting to urlcleaner hydrox 2006-08-21 06:42:42 +00:00
  • 135e019883 removed one superfluous line from last commit (hasnot is included in remove) orbiter 2006-08-21 01:59:44 +00:00
  • 1591a55963 added object cache miss-cache use for remove method orbiter 2006-08-21 01:51:27 +00:00
  • 8f3f4ab0eb enhanced synchronisation in plasmaWordIndex orbiter 2006-08-21 01:29:26 +00:00
  • f933f00f09 another patch to URL protocol handling for 'news', 'nntp' etc: reject it! (the java.net.URL class rejects them too) orbiter 2006-08-21 01:04:04 +00:00
  • 4c6e00d80a more bugfixes for URL class, see: http://www.yacy-forum.de/viewtopic.php?p=24844#24844 orbiter 2006-08-21 00:23:39 +00:00
  • 38c4248814 Some language updates Removes the ; behind Slovenky in language list rramthun 2006-08-20 11:52:02 +00:00
  • 23dd972608 fixed memory calculation in performanceMemory web page fixed also maximum cache size computation orbiter 2006-08-20 01:20:34 +00:00
  • b7dc251948 fixed bugs in url class: - correct backpath ('..') handling - correct absolute path handling - included https orbiter 2006-08-19 22:27:01 +00:00
  • 1ce3c22761 better memory control: - added memory monitor for preNURL-db in performanceMemory - changed default memory assignments orbiter 2006-08-19 13:09:04 +00:00
  • 39b4c26bdc more memory control: - catchup of OutOfMemoryError in server threads - automatic adoption of word cache size after a Short Mem Cycle orbiter 2006-08-19 00:06:39 +00:00
  • 3e9d509c39 some small fixes orbiter 2006-08-18 22:50:05 +00:00
  • 479861a3cf *) Value entered in searchfield will not get lost anymore when link "more options..." is clicked. If Javascript is disabled or function does not work for any other reason, there should be no change to current behaviour. low012 2006-08-18 21:53:15 +00:00
  • 276225d79e fix for URL class orbiter 2006-08-18 21:33:00 +00:00
  • 16993e84dd replaced non-7bit umlaut ascii @Franz: sorry, ich hab hier ein ant das daran rumzickt. Bitte in deinem Namen ein 'ss' statt der sz-Ligatur benutzen orbiter 2006-08-18 12:45:19 +00:00
  • eb633c0a4f server threads must now supply a method that can be called in case of short memory. This has been realized for the indexing thread. orbiter 2006-08-18 02:07:03 +00:00
  • f5720cb2fa removed most synchronization in wordIndex (for testing) orbiter 2006-08-18 01:35:33 +00:00
  • 0187c60010 because of a bug in the JRE 1.4.2 there was no memory protection see http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4686462 this commit fixes the bug by using a memory-computation patch. All uses of Runtime.maxMemory had been replaced by serverMemory.max The bug is not present any more in Java 1.5 orbiter 2006-08-18 01:33:54 +00:00
  • 4eca0f8830 *) fixed PPM calculation for multiple indexer-threads *) fixed totalPPM calculation and added total PPM to Network.html auron_x 2006-08-17 19:15:30 +00:00
  • cfb51fdef1 less synchronization in plasmaWordIndex orbiter 2006-08-17 00:10:50 +00:00
  • d6a928c2da quickfix for http://www.yacy-forum.de/viewtopic.php?t=2705 orbiter 2006-08-16 23:20:21 +00:00
  • 6ad471ef96 * applied many compiler warning recommendations * cleaned up code * added unit test code * migrated ranking RCI computation to kelondroFlex and kelondroCollectionIndex orbiter 2006-08-16 19:49:31 +00:00
  • cf1186597b utf fix from theli allo 2006-08-16 15:26:04 +00:00
  • 9add56dc09 *) Cursor gets placed in search field when page is loaded. If Javascript is disabled or function does not work for any other reason, there will be no change to current behaviour. low012 2006-08-15 22:21:33 +00:00
  • 34831d2d9f *) Check validity of crawl filter reg.exp. before adding it into the crawler queue See: http://www.yacy-forum.de/viewtopic.php?p=24671 theli 2006-08-14 16:11:22 +00:00
  • 9da3aa74d3 silly me, fix for the fix as advised by theli hydrox 2006-08-13 17:26:32 +00:00
  • bb3d9a5582 *) e.getMessage().indexOf() can only be used if there is actually an ExceptionMessage. hydrox 2006-08-13 17:18:09 +00:00
  • 7a54010a9c *) Iterators can't be casted to IndexContainer hydrox 2006-08-13 17:08:39 +00:00
  • 5e0b6f8f83 *) sorting peer name list on Blacklist_p.html *) restructuring of sharedBlacklist_p.java theli 2006-08-13 13:29:50 +00:00
  • 6c8366aea1 *) Bugfix for blacklist import function - wrong property name - list was accidentally imported into a new blacklist file theli 2006-08-13 09:20:43 +00:00
  • d0000a15d6 adjusted opensearchdescription to match requirements of A9 orbiter 2006-08-13 00:25:32 +00:00
  • cd5f7e137c fixed problem with NURL-generation upon first startup (a new kelondroFlexTable was generated, which should not) orbiter 2006-08-12 23:24:50 +00:00
  • 248627ab92 added open search compatibility orbiter 2006-08-12 17:59:13 +00:00
  • 8418af141a added several consistency checks and small changes orbiter 2006-08-12 15:59:14 +00:00
  • 9d13aeca13 *) removing class. does not work so far theli 2006-08-12 14:43:07 +00:00
  • 95a84ae469 *) adding missing classes theli 2006-08-12 14:41:26 +00:00
  • eee44be602 *) adding an interface for customized blacklist classes - now it's possible to use a customized blacklist engine instead of the default one - this can be done by configuring the property BlackLists.class See: http://www.yacy-forum.de/viewtopic.php?t=2108 theli 2006-08-12 14:28:14 +00:00
  • 6d2f15971a there is a very strange error that causes that the kelondroRecords structure is corrupted. The cause is, that the deleted-records-chain has wrong entries, and one of the pointers in that chain points to a place behind the file end. This causes an IndexOutOfBoundsException within an IO operation. I currently don't know the reason that the deleted-records-chain is corrupted, but the error can be catched. If this now happens with the assortment database, the database is deleted. See also: http://www.yacy-forum.de/viewtopic.php?p=24586#24586 orbiter 2006-08-12 13:45:23 +00:00
  • 79f27c298e *) next bugfix for blacklist - added items were not displayed properly - bugfix for disabled usage theli 2006-08-12 12:18:39 +00:00
  • 4abc04dac0 *) make Blacklist_p.html more failsafe if no blacklist file is available of the user has entered empty strings theli 2006-08-12 12:03:44 +00:00
  • 28c84e5b57 *) Bugfix for NullPointerException in Blacklist_p if blacklist directory is empty See: http://www.yacy-forum.de/viewtopic.php?p=24563 theli 2006-08-12 11:03:14 +00:00
  • 66f1eb07d9 *) Bugfix for IllegalArgumentException in transferURL See: http://www.yacy-forum.de/viewtopic.php?p=24560 theli 2006-08-12 10:54:19 +00:00
  • 1cab72c93c small changes reflecting the de.lng version (no author) 2006-08-12 10:22:39 +00:00
  • d2e8e76218 *) now it's possible to configure the yacy blacklist separately for dht, search, proxy, crawler See: http://www.yacy-forum.de/viewtopic.php?t=2541 http://www.yacy-forum.de/viewtopic.php?p=24516 theli 2006-08-12 02:42:10 +00:00
  • 9ae9062bd3 * disabled new kelondroFlex table for NURLs * added new RAM index Class * fixed possible synchronization problem in kelondroRecords orbiter 2006-08-12 00:58:43 +00:00
  • 689bbcf9cd replaced kelondroTree db for NURLs by new kelondroFlexTable The new database is only created if the old is deleted or does not exist orbiter 2006-08-11 23:36:58 +00:00
  • 7fbba41962 synchronization fixes orbiter 2006-08-11 23:04:36 +00:00
  • 328f9859a5 more synchronization in plasmaWordIndex orbiter 2006-08-11 22:07:59 +00:00
  • f43c90fa98 fixed handling of null referer in crawlOrder orbiter 2006-08-11 21:46:34 +00:00
  • 23a99b8283 Small changes to the language rramthun 2006-08-11 20:55:40 +00:00
  • 130e6d4719 generalized index object for eurl, nurl and lurl to prepare move of these tables to new kelondroFlexTable Object orbiter 2006-08-11 17:37:54 +00:00
  • acdf24877f more synchronization against outOfMemoryError in wordIndex orbiter 2006-08-11 16:27:56 +00:00
  • 95160d7f2c fixed size computation of index elements from the collection index orbiter 2006-08-11 16:01:18 +00:00
  • 26116cabde added missing rowdef assignment orbiter 2006-08-11 15:31:40 +00:00
  • cfbacbbf08 reverted change in robotsParser orbiter 2006-08-11 15:29:29 +00:00
  • abf22f6e60 removed url normalform computation from htmlFilterContentScraper. This method was implemented in de.anomic.net.URL orbiter 2006-08-11 15:09:22 +00:00
  • 740d49751d * strict type and size check in kelondroRow handling * adopted all code to use the declaration form of kelondroRow * fixed a bug in kelondroRow which caused wrong parsing of encoding type * the bug caused bad database behaviour in new indexCollection data structure. because of this bug, all test databases are now already void. A new database is created * the kelondroFlexTable and indexCollection data structures now store a declaration of the row definition into a properties file along the database files. orbiter 2006-08-11 03:20:44 +00:00
  • 314021453f * more logging * option in yacy.init to set useCollectionIndex usage orbiter 2006-08-10 21:21:50 +00:00
  • a52f36787f better templatedebugging allo 2006-08-10 14:02:03 +00:00
  • cbfba2026d removing the prefix option. allo 2006-08-09 17:51:20 +00:00
  • 3480d36417 added some debug code allo 2006-08-09 16:57:36 +00:00
  • 61b151b083 * added another auto-fix for collection index inconsitency check * fixed words size computation for collection index orbiter 2006-08-08 00:52:04 +00:00
  • 0bbbd129ef small fix for exception message orbiter 2006-08-07 23:52:12 +00:00
  • 718fbc2dae enhancements in kelondroCollectionIndex: * synchronized array and index objects * auto-fix function for slightly corrupted index entries * generalized internal access methods orbiter 2006-08-07 23:29:26 +00:00
  • ec5149ff3b fix for busyCacheFlush detection orbiter 2006-08-07 22:28:09 +00:00
  • f58283def2 better control of index flush orbiter 2006-08-07 22:07:17 +00:00
  • 4be21a3cab ups orbiter 2006-08-07 21:56:02 +00:00
  • 80b6c90d54 enhancements to prevent blocking during dht transfer receive orbiter 2006-08-07 21:49:39 +00:00
  • 4fb8fddd99 *)made the domainlist of the blacklist sorted if a new domain is added it is still appended to the end of the list and sorted in with next refresh, may need a fix. auron_x 2006-08-07 17:34:27 +00:00
  • 9f298083cd *) adding more urls to the error url - old error strings where replaced with there corresponding constants See: http://www.yacy-forum.de/viewtopic.php?t=2638 theli 2006-08-07 15:11:14 +00:00
  • d56f06401e - Cache known URLs during indexReceive to avoid getting blocked during loadedURL.exists() whenever possible - Small logging updates hermens 2006-08-07 11:42:00 +00:00
  • c09f734d06 *) offer router configuration on ConfigBasic.html - checkbox to allow router configuration is shown if - a) the UPnP forwarder is installed - b) a UPnP enabled router was found - c) no other forwarder was configured See: http://www.yacy-forum.de/viewtopic.php?p=24264 theli 2006-08-07 11:31:18 +00:00
  • dcbb4d0a6b Display the size of HashBlacklistedCache on PerformanceMemory page. hermens 2006-08-07 11:19:54 +00:00
  • c7b6389ca1 *) renaming indexDistribution.dhtReceiptLimitEnabled property to indexDistribution.transferRWIReceiptLimitEnabled so that the default value is taken over by all peers theli 2006-08-07 11:01:01 +00:00
  • 0baadcadca *) enable indexDistribution.dhtReceiptLimitEnabled limit per default See: http://www.yacy-forum.de/viewtopic.php?p=24425 theli 2006-08-07 10:51:24 +00:00
  • d799622da1 better flush limit for index collections orbiter 2006-08-07 00:44:43 +00:00
  • d468d665c9 some changes that may help to prevent deadlocks that cause an OutOfMemoryError as described in http://www.yacy-forum.de/viewtopic.php?p=24359 orbiter 2006-08-07 00:19:01 +00:00
  • 988341cf81 *) some comments added theli 2006-08-06 14:44:04 +00:00
  • d54767f634 *) last step of removing embedded html from dir class - migration finished *) dir list now sorts the dirlist entries. - directories are listed before files - files are sorted alphabetically, case insensitive theli 2006-08-06 14:38:07 +00:00
  • 96b774e427 Adding link to newsletters as agreed in forum. rramthun 2006-08-06 10:22:53 +00:00
  • 8283df2d77 *) first step of removing embedded html from dir class - dir list generation uses templates now theli 2006-08-06 08:09:39 +00:00
  • 279b1d969d Integrated new indexing data structure 'collections' into the main class for indexing, the plasmaWordIndex. orbiter 2006-08-05 22:22:14 +00:00
  • 4ff742e42d implemented indexCollectionRI this is the new database structure that is supposed to replace the plasmaAssortmentCluster AND the plasmaWordIndexFileCluster The new structure is not yet active and needs to be integrated into plasmaWordIndex. This has some migration constraints that are not yet completely solved. orbiter 2006-08-05 19:18:33 +00:00
  • 132cd7da45 no need to copy dir.* allo 2006-08-05 18:39:25 +00:00
  • 0164321160 fix for the actions (uploading/deleting, loggin in, ...) allo 2006-08-05 18:34:31 +00:00
  • 01f95eccd3 re-write of kelondroCollectionIndex. This is the data structure that shall replace the current assortment files. * used the kelondroFlexTable to hold the index of collections * used kelondroRow definitions to declare all data structures * fixed several bugs that appeared in kelondroRowSet and kelondroRowCollection during testing orbiter 2006-08-04 23:04:03 +00:00
  • ebc2233092 * implemented (finished) class indexRowSetContainer * replaced indexTreeMapContainer by indexRowSetContainer * deleted indexTreeMapContainer and abstract class This is another step to the new database structure orbiter 2006-08-02 23:20:03 +00:00
  • 9183d21f25 renamed new index class to old name orbiter 2006-08-02 20:01:59 +00:00