Commit Graph

  • 432154f725 new strategy for concurrent database index key retrieval orbiter 2009-09-29 08:04:00 +00:00
  • 09775daa60 slightly changed my skin (vega-aqua): - search result headline font is slightly larger and has an underline (like g**gle) suessthomas 2009-09-28 19:50:43 +00:00
  • a11cd9f80f - removed reverse name lookup for http access logging (grr..) - removed a synchronization in seed info string generation orbiter 2009-09-28 15:23:15 +00:00
  • 2e6bdce086 - added more logging to balancer - changed balancer logic slightly orbiter 2009-09-27 22:35:22 +00:00
  • 3c4064932c *) added width and hight to prevent the page from "jumping" when the image is reloaded automatically in Opera 10 low012 2009-09-26 22:32:52 +00:00
  • 5e4f267a36 *) added subversion properties and edited a few comments low012 2009-09-26 22:07:40 +00:00
  • 2d01411bbc *) tested string will appear in input box now (when testing a similar URL it does need to be copied anymore, it can be edited it right away) *) https:// and ftp:// can be used as start of the string to be tested now too *) better error handling (no text in Java anymore) low012 2009-09-26 21:33:33 +00:00
  • 3faa011e3d added another search integration help page orbiter 2009-09-25 14:03:41 +00:00
  • 26b81bd1f1 added another search integration help page orbiter 2009-09-25 14:03:07 +00:00
  • 69a091de17 added skin for geocaching search portal orbiter 2009-09-24 22:53:50 +00:00
  • 1171a72006 fix for deadlock as seen in http://forum.yacy-websuche.de/viewtopic.php?p=17521#p17521 orbiter 2009-09-24 19:14:35 +00:00
  • 031e6eefbd some updates to dublin core, metadata browsing, file indexing and parser stability orbiter 2009-09-24 12:54:45 +00:00
  • 62a7341c4d Fix for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2204 hermens 2009-09-24 11:38:15 +00:00
  • f65bfaa9af *) Removed base tag from errror page. This has been added by myself a long time ago as a workaround for some weird behavior of my router, but as it turns out, it does more bad than good in general: If HTTPS is used for communication with YaCy, entering a wrong passwort led to an errror page with a form which would send username and password unencrypted with the user possibly being unaware of this. *) changed some comments, added some annotations, added SVN properties here and there low012 2009-09-23 21:26:14 +00:00
  • e4797ebcde fix for http://forum.yacy-websuche.de/viewtopic.php?p=17509#p17509 corrupted files are ignored orbiter 2009-09-22 22:00:45 +00:00
  • efa7fb34f0 better oom-awareness of miss-cache in cache orbiter 2009-09-22 15:01:29 +00:00
  • 3e9dcfc204 fix for http://forum.yacy-websuche.de/viewtopic.php?p=17504#p17504 orbiter 2009-09-22 14:39:06 +00:00
  • c3a4aee255 some redesign with a possible fix for the ReferenceContainerCache. orbiter 2009-09-22 14:33:57 +00:00
  • e5d04f6b91 added vega's skin see http://forum.yacy-websuche.de/viewtopic.php?p=3530#p3530 orbiter 2009-09-21 22:06:15 +00:00
  • feece4bfcb slightly changed default skins: - search result headline font is slightly larger and has an underline (like g**gle) - no dashed line between results in grey style - no search results with menu on left side by default (but is still possible) orbiter 2009-09-21 21:30:24 +00:00
  • aca8a78eb8 fix for shutdown of DocumentIndex objects orbiter 2009-09-21 15:24:47 +00:00
  • 23ab6fbca4 - navigation appear at correct position when opengeodb-results are also presented after a search - show an about box if about.headline and about.body is set orbiter 2009-09-20 22:10:45 +00:00
  • 4db34eea73 fix for OOM problem in kelondro Cache orbiter 2009-09-19 07:50:46 +00:00
  • 8ea1d7ab59 fix for wrong assert condition in search abstract generation orbiter 2009-09-18 21:49:38 +00:00
  • 70bda620db another fix for/against bad debian packages orbiter 2009-09-18 21:43:23 +00:00
  • fbd77bd77c git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6328 6c8d7289-2bf4-0310-a012-ef5d649a1542 orbiter 2009-09-18 12:33:35 +00:00
  • 4652b4d299 fixed debian package orbiter 2009-09-18 12:30:22 +00:00
  • 54c7cbf1d9 - fast result for local search in case that less than 10 hits exists - small change in display of RAM in profiling orbiter 2009-09-18 09:19:52 +00:00
  • 28d4b921b6 different approach for file search orbiter 2009-09-18 07:54:16 +00:00
  • f99f86c5c5 added concurrency to file indexing class orbiter 2009-09-17 22:37:21 +00:00
  • 902d16cf6c fixes to parser orbiter 2009-09-17 22:00:19 +00:00
  • 4a1c852435 fix in usage of RAM copy for Table objects and some cosmetics in asserts. This bug affected Tables in case that a removeOne() was called and a RAM copy of the table was active. It may happen for peer owners with a lot of RAM assigned to YaCy. The bug appeared especially during crawling when the balancer tried to get new entries from the crawl queue. This bug may help to solve report at http://forum.yacy-websuche.de/viewtopic.php?p=17417#p17417 and will be tracked there orbiter 2009-09-17 20:41:50 +00:00
  • 11323bcb36 added debug flag to built to enable better debugging of yacycore.jar orbiter 2009-09-17 19:21:07 +00:00
  • dce450e2e0 possible fix for "hung" doc-documents lotus 2009-09-17 17:08:32 +00:00
  • a4888249f1 Small improvements in the German language. My first amendment, I hope it still works all ... suessthomas 2009-09-16 20:33:24 +00:00
  • db1b030276 fixed class path in startup scripts; omitting old classes path; all classes are now in lib/yacycore.jar orbiter 2009-09-15 17:29:18 +00:00
  • 033554dc12 more logging in tranferURL to find OOM bug. See also http://forum.yacy-websuche.de/viewtopic.php?p=17361#p17361 orbiter 2009-09-15 12:59:55 +00:00
  • e627f75415 one more fix to badwords and stopwords orbiter 2009-09-15 11:47:50 +00:00
  • 721b88efbd - fixed a problem loading blacklists with new yacycore.jar - fixed badwords and stopwords initialization orbiter 2009-09-15 11:46:02 +00:00
  • 80d5005044 fixed seed upload methods - replaced reflection with direct instantiation orbiter 2009-09-15 09:47:02 +00:00
  • 68465c37af added a convenience class to add files into a YaCy index to make this possible, the yacyURL must be able to process file:// urls, which has also been implemented testing of the new class resulted in some bugfixes in other classes orbiter 2009-09-14 21:17:42 +00:00
  • 2e41e10ffd - updates to yacyVersion parser (remove old targets) - added javadoc target to built script (does not work yet without errors) orbiter 2009-09-14 18:08:40 +00:00
  • 1ddaf16466 added generation of a yacycore.jar yacy library file: when a ant dist is envoked, the release is built using the new yacycore.jar without the class files in classes YaCy consists currently of two main parts: the YaCy core and the YaCy interface. This change will make it possible to use the YaCy core as part of other projects. orbiter 2009-09-14 17:06:09 +00:00
  • 5658044388 - when there are no topics, the topics window is not shown - othervise: close topics window by default There will be more navigation in the future (i.e. update-time selection) and the topics will be less important than the others, therefore it is now hidden by default orbiter 2009-09-11 14:54:53 +00:00
  • 27d00285aa - added a new file reader cache that may serve as full-file-copy of blob database files. This is not yet used - removed class FileWriter and replaced all usage of that class with CachedFileWriter orbiter 2009-09-09 21:28:23 +00:00
  • fd6b9cb7dc refactoring of IO access classes orbiter 2009-09-09 20:49:16 +00:00
  • d64569aa39 reuturn only recommendations of words that have a greater count than the original word orbiter 2009-09-08 13:57:52 +00:00
  • 604c37927f used comparator for did-you-mean that uses index sizes for comparisment, but: - limit comparisment to only the first 10 elements that had been sorted before without IO - added a size cache to index computation because the size is computed at least twice in set comparator orbiter 2009-09-08 13:48:17 +00:00
  • a58d9cae7d - show location name in geolocalization search result - added link from location icon to openstreetmap browser with coordinates orbiter 2009-09-08 10:18:03 +00:00
  • 573d03c7d7 added configuration to enable ram table copy orbiter 2009-09-07 20:30:57 +00:00
  • 8274251467 *) stopped page content from jumping in Opera 10 when loading an image took too long low012 2009-09-07 20:19:26 +00:00
  • 3be54e1891 fix to rule when to use a ram table copy orbiter 2009-09-07 19:22:12 +00:00
  • 700218846c disabled or removed sleep calls orbiter 2009-09-07 18:50:44 +00:00
  • 342c5d0fd4 fixed city name detection: finds now also substrings of city names orbiter 2009-09-06 20:44:02 +00:00
  • 18aa0609ca fix for caching of word hash computation orbiter 2009-09-05 22:05:18 +00:00
  • a10a6cce45 patch for http://forum.yacy-websuche.de/viewtopic.php?p=17289#p17289 orbiter 2009-09-05 21:52:50 +00:00
  • 53bbdfd19a *) setting SVN keywords *) minor changes low012 2009-09-05 20:41:21 +00:00
  • 25f6145934 *) preventing null pointer exception in case empty search word or only one character is enterd or all search words are removed by filters low012 2009-09-05 20:31:39 +00:00
  • 248f3fd9b5 *) cleaned up code for better readability *) added a few copyright notices *) removed redundancy in constructors of ListToken low012 2009-09-05 20:04:23 +00:00
  • eaddf2d464 - corrected layout of map preview - added caption to maps containing latitude and longitude information - prevented that maps occur on second search page - added location names to did-you-mean - some refactoring of did-you-mean - added equal and compareTo test to Coordinates class to make that work in set - fixed utf-8 support for library files - fixed a bug in images search icon view caption orbiter 2009-09-04 23:33:47 +00:00
  • 4b83875abd Small fixes for the heapCacheIterator in ReferenceContainerCache: - Start the iteration at startWordHash - When used with rotation, let the iteration stop when the cache is empty hermens 2009-09-04 23:27:14 +00:00
  • fd668f531b fixed map layout orbiter 2009-09-04 19:20:27 +00:00
  • 2740d9dd79 added integration of osm maps for search orbiter 2009-09-04 14:32:36 +00:00
  • af3a696fc4 added a fast-fail concept in search processes. The search now has better control if all the remote searches may bring any result. If all processes are finished, then all search tasks fail fast. orbiter 2009-09-03 23:09:53 +00:00
  • 847c3027ff added - a time-out of 10 seconds - and a clear-on-error in yacyui-portalsearch, to make the loading bar disappear when nothing is found. orbiter 2009-09-03 22:13:04 +00:00
  • ce972ff4ef update to default ranking profile which has now some settings to deny some phpbb3 pages which are redundant in the index when crawling phpbb3. orbiter 2009-09-03 20:54:47 +00:00
  • 44579fa06d - fixed a problem loading images through yacy's document loader, this denied non-parseable documents which excluded all images - fixed url of osm tile server orbiter 2009-09-03 11:46:08 +00:00
  • 67eddaec4b changed way to integrate dictionary files: the must be downloaded manually by the user and placed in DATA/DICTIONARIES/source for each externally imported dictionary file there will be a translator that converts the input file once into a YaCy-internat data format. Files that will be provided together with yacy releases may still be placed in <root>/dictionaries orbiter 2009-09-02 18:42:13 +00:00
  • d656a94f55 fix for bad paths in dictionary processing orbiter 2009-09-02 18:24:41 +00:00
  • 3b9aaf9e9f - inserted new library tests inside DidYouMean - some redesign of DidYouMean that was necessary to follow a special rule how a library should be used: - the library provides words that start or end with a test word which may be possibly also an empty set of words - all words that the DidYouMean produced with the four production rules are used to generate a set of library-completed words - if this process results in any words from the library, only library-genrated words are taken - if the is no library-generated word at all, take the artifial generated word - all words that result from these rules are tested against the index - the result is ordered using a lightweight comparator that prefers short words - a not-so-much-io test against the index is beeing prepared next - insered the library initialization into the switchboard orbiter 2009-09-02 13:41:56 +00:00
  • 8c35ffe34c fixes to the dymlib orbiter 2009-09-01 15:05:28 +00:00
  • bfa273bcc1 added a library provider which holds libraries in static objects, which can be used by any other classes to support their functions. libraries are designed in such a way that users can create and insert their own library files, but can also be imported from other sources. As an example the "Korpusbasierte Wortgrundformliste DeReWo des Institut für Deutsche Sprache" from http://www.ids-mannheim.de has been integrated. This dictionary is licensed to be used for all non-profit purposes. In case that YaCy is used for commercial uses, this library must be removed. The new library provilder reads the original source and translates it into a simple word list to be used for the did-you-mean library provider. More libraries may be provided in the future using a download-servlet which puts files from the internet into the <application-root>/dictionaries/ path. orbiter 2009-09-01 13:54:43 +00:00
  • 1762a7bcd6 - moved DidYouMean to the data package - added a DidYouMeanLibrary class that shall support the did you mean function with additional word lists orbiter 2009-09-01 13:04:35 +00:00
  • bf8ed00e9e removed debugging code orbiter 2009-08-30 11:03:34 +00:00
  • ead48c4b25 fix for preparation of search result pages with offset > 10: - less pages are fetched in advance - just-in-time fetch of next required pages - fix for missing hand-over of offset to fetch threads orbiter 2009-08-30 10:28:23 +00:00
  • 39a311d608 better care to do not loose the merge/dump thread orbiter 2009-08-29 23:35:03 +00:00
  • 10d3e856b5 better concurrency, less blocking & performance hacks orbiter 2009-08-29 23:34:14 +00:00
  • 1a9cfd8718 some performance hacks (CPU only, not IO) this will cause better computation speed for single- and multi-core; there are enhancements that will speed up old and slow machines as well as multi-core CPUs. Indexing of surrogates has been speed up from 4000 PPM to over 20000 PPM on a simple dual core office computer. Since the enhancements are mostly in core routines, the hack should also speed up search performance. orbiter 2009-08-28 13:28:11 +00:00
  • 92407009b2 cleanup orbiter 2009-08-27 23:20:59 +00:00
  • 0ba1beaf56 separated rwi constraint evaluation from rwi ranking and added concurrency orbiter 2009-08-27 22:54:32 +00:00
  • ce7924d712 better concurrency for rwi entry parsing during search processing orbiter 2009-08-27 22:06:52 +00:00
  • b0637600d5 enhanced url constraint computation: better position of constraint check during retrieval process orbiter 2009-08-27 20:20:07 +00:00
  • 3ebb228ea1 added smaller icon for widget orbiter 2009-08-27 19:44:39 +00:00
  • 61748285c3 more refactoring of search orbiter 2009-08-27 15:19:48 +00:00
  • 323a8e733d removed unused classes orbiter 2009-08-27 14:42:05 +00:00
  • 72e5407115 refactoring of snippet cache orbiter 2009-08-27 14:34:41 +00:00
  • 0e471ba33b - fixed a bug in fast digest computation - added a open-on-demand hack to heap files: when a heap file is opened the first time, it is first scanned to get a key index and then it is closed again. This will free up file pointers in cases where a really large number of blob files are opened upon initialization of ArrayStack objects. This should solve also a problem reported in http://forum.yacy-websuche.de/viewtopic.php?p=17191#p17191 orbiter 2009-08-27 11:03:21 +00:00
  • 93b2622503 *) repaired and added IM online status indicators *) added some missing SVN properties *) removed unnecessary comment, added missing copyright notice low012 2009-08-26 18:34:00 +00:00
  • e7736d9c8d more refactoring: made all variables in SearchEvent private to prepare splitting of the class into two parts: local and remote search orbiter 2009-08-26 15:59:55 +00:00
  • 4b92d0b9b7 patch for possible problems with normalization of '/' in urls. This applies in rare cases when '/' appear in post-properties orbiter 2009-08-26 15:10:03 +00:00
  • d8ca6e6bf1 more refactoring for search orbiter 2009-08-25 21:27:01 +00:00
  • becb30fa12 changes by Thomas Süß daburna 2009-08-25 09:19:50 +00:00
  • fe4a4e3f6b added missing class orbiter 2009-08-24 21:03:40 +00:00
  • 72ac5bd80f refactoring of search process. this is the beginning of some architecture changes that will hopefully bring some more stability, speed and transparency to the search process. orbiter 2009-08-24 15:24:02 +00:00
  • c4d0e22a77 Further speed upof concurrent DHT-receive hermens 2009-08-23 22:37:15 +00:00
  • 2fbc0696bf Fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2334 hermens 2009-08-21 13:10:59 +00:00
  • 07f505154d updated German translation daburna 2009-08-19 13:58:40 +00:00
  • d515bc11e2 added ooxmlparser f1ori 2009-08-08 15:34:41 +00:00
  • 8c1b02af04 * fix warning in testcase f1ori 2009-08-08 15:18:02 +00:00
  • d9744b1b5d replaced old caching strategy control class with lightweight simplearc orbiter 2009-08-07 23:01:33 +00:00