Mirrors/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2025-07-21 09:04:37 -04:00
Code Issues Packages Projects Releases Wiki Activity

Commit Graph

  • 698730c4eb better utf-8 support everywhere using a default utf-8 encoding for all string conversions. todo: add this also for debian start script (how do I do that?) orbiter 2010-09-03 22:18:10 +00:00
  • ca0a03e9ea ... migrating to HttpComponents-Client-4.x ... ssl-stuff: accept almost everything sixcooler 2010-09-03 16:02:52 +00:00
  • d8f52c5b9c added a changelog url to download orbiter 2010-09-03 12:55:36 +00:00
  • 1da5241c2d do not block server session if maximum number of sessions is reached, just try to clean up once orbiter 2010-09-03 12:05:37 +00:00
  • 3988a95fb5 added ability in rss reader to parse atom feeds orbiter 2010-09-03 08:53:24 +00:00
  • 5de70c3d7c changed way of storage for search requests: - the search request cache can now get as large as 1000 entries - if more entries arrive, unused are deleted - the elements may stay in the cache up to 10 minutes and longer if they are used - the elements are deleted earlier that 10 minutes if the memory gets low This commit was mainly done for metager-feeding peers that have a query load of 50000 queries each day. Also added: - a monitor for cache hit/cache miss in PerformanceMemory_p.html (see at bottom of page) orbiter 2010-09-02 21:52:45 +00:00
  • 9d080f387e change in handling of the all-visible home path for storage in YaCy: the home path can now be distinguished between - data home; the path where the DATA directory is created - application home; everything else This will make it possible to store application data on Mac releases within the ~/Library/YaCy directory; a place where Mac applications write their data. Similar techniques will be possible for debian and windows. To use the new data path, YaCy can be started with -start <data path> or -gui <data path> orbiter 2010-09-02 19:24:22 +00:00
  • fa5683adfe create a mac dmg file (a disc image) for mac releases in ant orbiter 2010-09-02 19:11:49 +00:00
  • 875741bcff fix for http://forum.yacy-websuche.de/viewtopic.php?p=20657#p20657 orbiter 2010-09-02 10:05:04 +00:00
  • 091281c9f2 Mac app ant task building a ready-to-distribute zip file extending r7080 lotus 2010-09-02 08:01:01 +00:00
  • 65eaf30f77 redesign of crawl profiles data structure. target will be: - permanent storage of auto-dom statistics in profile - storage of profiles in WorkTable data structure not finished yet. No functional change yet. orbiter 2010-08-31 15:47:47 +00:00
  • 3f1d5a061f by default store crawled pages to HTCache to support verify=false snippet generation orbiter 2010-08-31 09:28:01 +00:00
  • 2009999162 show landing page after installation finished lotus 2010-08-30 20:04:19 +00:00
  • 938676265f fix shutdown command, close HttpClient connection pool f1ori 2010-08-30 17:48:20 +00:00
  • 55da979291 disable revision detection for git f1ori 2010-08-30 17:11:19 +00:00
  • 6d2e0f5fb4 always kill shutdown java instance, even if yacy succeeded, in future, the TERM-signal should be used, but currently not all threads are joined during shutdown f1ori 2010-08-29 23:26:03 +00:00
  • be0abd92cd always use kill command in initscript, after timeout elapsed and yacy didn't finished f1ori 2010-08-29 18:15:22 +00:00
  • 2a4ddc48bb adjustment for new java download method see http://forum.yacy-websuche.de/viewtopic.php?p=20616#p20616 lotus 2010-08-27 18:55:44 +00:00
  • e9160ea1e5 Mac ant task according to r7023 lotus 2010-08-27 18:40:32 +00:00
  • 93d2c22e60 adapted memory for first run to current standard values lotus 2010-08-27 18:38:02 +00:00
  • 104318d58a - added nice colors to feed indexing state messages - added a 'remove all' button for new and scheduled rss feed list - made adding of new rss feeds concurrent so interface is more responsible orbiter 2010-08-27 11:56:51 +00:00
  • 23ba107834 UPnP port forwarding default on now. This also displays a message on the entry settings page if not successful, so the user gets an extra hint to open his ports. lotus 2010-08-27 08:45:00 +00:00
  • d5ccbb99f9 the Windows installer now always requires admin level for installation (Vista/7) unfortunately some users seem to forget to manually install the downloaded Java runtime and therefore could not start YaCy - added concept to always distribute the latest Java version via external php script lotus 2010-08-26 16:53:20 +00:00
  • 4f22e2df41 bugfixes for - next-execution-time in scheduler - deletion of scheduled rss feed loading (now deletes also the scheduling entry) orbiter 2010-08-26 16:42:00 +00:00
  • 42414a6ae3 added two more tables in rss reader interface: - fresh recorded rss feeds (not yet loaded or in scheduler) - rss feeds in scheduler The first list has a button that can be used to place rss feeds into the scheduler The second list has a button to delete rss feeds from the scheduler orbiter 2010-08-26 16:01:45 +00:00
  • 0010cd9db1 Support for indexing of RSS feeds! - added a scanning in html parser for rss feeds - storage of rss feed addresses, can be viewed with http://localhost:8080/Tables_p.html?table=rss - rss items retrieved by http://localhost:8080/Load_RSS_p.html (in Index Creation menu) can be selected and indexed - a rss feed retrieved in http://localhost:8080/Load_RSS_p.html can now be fully indexed - indexing of rss feeds can be placed in scheduler orbiter 2010-08-25 18:24:54 +00:00
  • 0f276dd63f - MapHeap now implements Map<byte[], Map<String, String>> - refactoring of method names to comply with Map method names orbiter 2010-08-24 12:36:56 +00:00
  • cf07b34c2d implemented the Map interface in the ARC classes so it will be possible to instantiate ARCs as Map<byte[], Map<String, byte[]>> Because such Maps with byte[] keys cannot be stored in hash maps (bad hashing on byte[]) another ARC with comparable Maps has been added orbiter 2010-08-23 23:38:03 +00:00
  • c60d0282fd more abstraction for tables stored in heaps: the BEncodedHeap now implements Map<byte[], Map<String, byte[]>> This will make it possible that also different database storage types may be added that implement also the same Map<byte[], Map<String, byte[]>> interface. orbiter 2010-08-23 21:27:58 +00:00
  • d1be64d491 removed wrong assert orbiter 2010-08-23 21:02:28 +00:00
  • 3197ca42ed preparations to move the HTCache into cora: - move the header framework classes to cora - move the ARC caching classes to cora - refactoring of code to call these classes from cora orbiter 2010-08-23 12:32:02 +00:00
  • 844f158686 - removed dependencies in header framework: moved http date methods from DateFormatter to HeaderFramework changed logging to log4j - added ftp load access to MultiProtocolURI - ensured termination of RSS feed iteration orbiter 2010-08-23 11:41:12 +00:00
  • 80ba543d4c svn fix for uppercase problem orbiter 2010-08-23 01:16:17 +00:00
  • 5e7081cd19 refactoring towards a unified loading mechanism for MultiProtocolURIs orbiter 2010-08-23 01:08:56 +00:00
  • caece04f26 removed System.err and System.out usage from FTPClient; changed logging to log4j (preferred in yacy.cora) orbiter 2010-08-22 22:51:31 +00:00
  • 90531f78ff refactoring of the cora package to get subpackages for http and ftp (smb to come) orbiter 2010-08-22 22:32:39 +00:00
  • d0fb6bc2bc cleaned up superfluous classes after sixcoolers migration to HttpComponents-Client-4.x orbiter 2010-08-22 22:04:31 +00:00
  • dcd9065c84 next try to fix loading of network picture orbiter 2010-08-22 22:02:54 +00:00
  • 661867923a ... migrating to HttpComponents-Client-4.x ... The Client is dead, long live the Client! (no references to the old client) sixcooler 2010-08-22 17:38:27 +00:00
  • 6e4d2f0800 fix for the network image sync bug orbiter 2010-08-21 10:59:21 +00:00
  • 7aa860c505 - more logging - more stability for database heap in case of buffer failure orbiter 2010-08-21 10:16:05 +00:00
  • 4d5446d641 code cleanup orbiter 2010-08-21 00:08:36 +00:00
  • 6b06e94c8c make searched word(s) in search-results viewable at dark themes sixcooler 2010-08-20 23:54:46 +00:00
  • 66ac3a7d9d corrected database row iteration orbiter 2010-08-20 23:33:56 +00:00
  • dfd416e3fb removed a mysterious image buffer orbiter 2010-08-20 23:13:59 +00:00
  • e10cd115a9 - added a new RSS reader interface. This is not finished but you can now load and look at RSS feeds. It will be used to index RSS feeds in a way that is appropriate for such kind of data. - refactoring of Mediawiki and PHPBB3 loader interface names (just renamed) - removed two old not used RSS loader interfaces - fixed a bug in RSS parser library of cora - added a new RSS parser component to the set of yacy document parsers orbiter 2010-08-20 11:30:02 +00:00
  • 933dc1a600 removed old rss parser (will be replaced with parser from cora package) orbiter 2010-08-20 07:42:38 +00:00
  • 70dd26ec95 added the new crawl scheduling function to the crawl start menu: - the scheduler extends the option for re-crawl timing. Many people misunderstood the re-crawl timing feature because that was just a criteria for the url double-check and not a scheduler. Now the scheduler setting is combined with the re-crawl setting and people will have the choice between no re-crawl, re-crawl as was possible so far and a scheduled re-crawl. The 'classic' re-crawl time is set automatically when the scheduling function is selected - removed the bookmark-based scheduler. This scheduler was not able to transport all attributes of a crawl start and did therefore not support special crawling starts i.e. for forums and wikis - since the old scheduler was not aber to crawl special forums and wikis, the must-not-match filter was statically fixed to all bad pages for these special use cases. Since the new scheduler can handle these filters, it is possible to remove the default settings for the filters - removed the busy thread that was used to trigger the bookmark-based scheduler - removed the crontab for the bookmark-based scheduler orbiter 2010-08-19 23:52:38 +00:00
  • 5a994c9796 added a scheduler based on API actions - every process that is monitored with the API Steering interface can now be scheduled! - added input methods in Steering interface to set a scheduling time - added a view on the steering api that shows only crawl jobs inside the Crawl Profile servlet - added a scheduling call process in the cleanup process handler that triggers the scheduled processes This causes that the cleanup now also looks for scheduled processes. Such processes are therefore not executed at the same time as given in the target execution time but they will be executed within the cleanup process time window. orbiter 2010-08-19 12:13:54 +00:00
  • 189a986ebd - modified api-call interface to record api calls with references to api-call database (carries pk) - added recording date, last execution date and next execution date for a scheduler (scheduler to be implemented next) - extended database access methods for more data formats, especially for date insert/retrieval - extended 'Steering' interface to show new database fields - migrated Steering to new http client - extended cora http client to transmit authentication and also added some convenience methods (http response code) - simplified database back-end (not so much specialized methods for multiple properties) - extended date formatter to produce a special format to show dates in html (&nbsp; in spaces of date format) orbiter 2010-08-18 15:56:38 +00:00
  • 1bc08e1416 support debconf in debian package * now you are ask some questions to preconfigure yacy after installing the debian package f1ori 2010-08-18 13:30:57 +00:00
  • 054c22e2c6 added TLDs from http://www.opennicproject.org orbiter 2010-08-18 10:39:49 +00:00
  • f616cdfce4 better resistance of NetworkImage generation against heavy load this is needed for the network image on the yacy.net home page orbiter 2010-08-18 09:51:00 +00:00
  • 2f8ff8ec02 de.lng: Added some German translation for Config* pages that I have found untranslated ConfigNetwork_p.html: Updated Javascript for P2P <-> Robinson selection to use the new ID values - sorry for breaking this in 6996 (undoing id -> name changes again in 7041 and 7042 because the name tag is not allowed in XHMTL Strict 1.0) mikeworks 2010-08-18 05:55:11 +00:00
  • 86d7f8a989 - the web visualization can now be generated in custom color - added input fields in WatchWebStructure_p.html - introduced enum classes for Draw Mode and Filter Mode orbiter 2010-08-17 10:44:00 +00:00
  • 64d4204f44 fix for NPE in network image computation orbiter 2010-08-17 08:18:17 +00:00
  • 7fdb17bb96 redirect uncaught exceptions to logging + small other changes orbiter 2010-08-16 12:33:06 +00:00
  • 237cfc44b0 fixed auto-set values for robinson selection; this reverts a single line from SVN 6996 orbiter 2010-08-15 18:23:47 +00:00
  • 85501414e9 Amending and supplementing de. Translation of the network.html suessthomas 2010-08-14 20:11:23 +00:00
  • 239fec1c42 Small Changes suessthomas 2010-08-13 20:49:22 +00:00
  • 59c035c40b changed explanation of Xmx and Xms orbiter 2010-08-13 20:03:10 +00:00
  • 92df768c39 * fix http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2929&hilit= * strings for navigation links have to be urlencoded f1ori 2010-08-13 15:59:52 +00:00
  • eb97bed1df patch for http://forum.yacy-websuche.de/viewtopic.php?p=20576#p20576 orbiter 2010-08-13 13:06:06 +00:00
  • 87b1684211 additional double-check in balancer orbiter 2010-08-12 17:18:15 +00:00
  • 653e617a1f * fix debian package dependencies for recent ubuntu and debian testing f1ori 2010-08-12 15:05:10 +00:00
  • 0d81731e88 fixed crawler bug caused by NPE in logging orbiter 2010-08-12 01:29:56 +00:00
  • a82a93f2fc - better url double check in crawler - more logging for error urls orbiter 2010-08-11 09:54:18 +00:00
  • a6ed6e8cb9 ... migrating to HttpComponents-Client-4.x ... make the occurrence of multiple header-keys possible sixcooler 2010-08-10 21:22:30 +00:00
  • 171f2bd84e - removed unused network oanet - added new network definition 'allip' which can be used in networks where intranet and internet-addresses shall be indexed - added a auto-switch-off for global search if there are no global peers orbiter 2010-08-09 23:41:17 +00:00
  • 1802c54317 LGPL-Header sixcooler 2010-08-09 14:38:49 +00:00
  • a835a22b32 fixed isLocal() property (better recognition of intranet hosts) orbiter 2010-08-09 11:22:56 +00:00
  • b480b7a4d0 fix for bug in last commit orbiter 2010-08-09 00:13:32 +00:00
  • b12bfe1f91 better usage of OSM tile cache and YaCy cache by usage of better tile server computation based on a coordinate hash orbiter 2010-08-08 23:51:37 +00:00
  • 388aa021c2 - concurrent loading of OSM tiles - added a 4-time re-try in case that tile server does not respond orbiter 2010-08-08 23:14:08 +00:00
  • 670c746dc5 dual-licensed HttpConnectionInfo for LGPL original GPL license holder granted dual-licensing by email orbiter 2010-08-07 23:03:10 +00:00
  • 838e8a1cb7 added a Macintosh OSX application wrapper. You can now create a YaCy Mac application like this: - build a new version of YaCy - take everything that is in the yacy home directory (from the fresh build) and move it into a copy of the YaCy.app directory under YaCy.app/Contents/Resources/Java/ - ready: double-click the YaCy.app icon and the new YaCy gui starts, then the browsers is also opened. - if the Mac application is terminated, also the YaCy demon should be stopped automatically and vice versa. This is currently not really good working and needs some more care. A build script for this is missing. Does anyone know how to do that in ant? orbiter 2010-08-05 11:11:15 +00:00
  • 301a59e07f moved browser access method from kelondro/util/OS to gui/framework/Browser orbiter 2010-08-05 10:49:58 +00:00
  • ec72387165 added a very early test version of a YaCy gui component. The gui currently does nothing else than providing a search window that sends the search string to the browser The gui is started when YaCy is started with the option -g or --gui, like ./startYACY.sh -g The gui will primary be used to provide a 'real' macintosh version that can be started and operated like any other macintosh application. A special mac application wrapper will follow. orbiter 2010-08-05 10:43:03 +00:00
  • d88b9606d1 fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2923 + some client fine tune sixcooler 2010-08-04 16:58:33 +00:00
  • 6388a58fc7 better memory management and slightly less (in total and temporary) RAM allocation: - confirm that database objects that are not supposed to grow do not have a index memory management that is designed for growth - changed index sorting method in such a way that it allocates less objects during quicksort - database classes classes renaming (shorter, naming addresses that objects hold in RAM) - added a large number of asserts to check if objects actually take the RAM that they should have orbiter 2010-08-04 13:33:12 +00:00
  • 5924a0d851 - enhanced concurrency in database index access for multicore - added statistics about database index caches in PerformanceMemory_p.html - adoped many classes to use the new statistics - added missing close statements orbiter 2010-08-03 04:58:48 +00:00
  • 55a2536bcf enhancement in drawing speed and reduction of object allocation during drawing orbiter 2010-08-03 02:44:08 +00:00
  • 610855e362 do not use network graph cache if called from authorized account orbiter 2010-08-03 02:43:15 +00:00
  • 9ab06bc333 enhancement in sorting efficiency (database root operation): less object allocation orbiter 2010-08-03 02:42:28 +00:00
  • 39d96abbb5 fix yacyRelease download (http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2920&p=20545#p20545) better cookie policy sixcooler 2010-08-02 20:13:20 +00:00
  • 1bfa21f973 *) HTML fix low012 2010-08-02 17:31:05 +00:00
  • 349e4dee9d ... migrating to HttpComponents-Client-4.x ... added cookie policy sixcooler 2010-08-02 14:16:44 +00:00
  • c29f24a519 ... migrating to HttpComponents-Client-4.x ... - Proxy - Release-download sixcooler 2010-08-01 22:35:11 +00:00
  • ced07970c1 *) fix for last commit *) HTML fixes low012 2010-08-01 19:04:57 +00:00
  • 4e60c69f84 *) HTML fixes low012 2010-07-28 15:54:44 +00:00
  • 66266a288e better network image cache orbiter 2010-07-28 14:54:56 +00:00
  • e7ea3b3cc5 added a buffer for network images to reduced load on yacy.net network image server orbiter 2010-07-28 12:45:53 +00:00
  • d5c65b17a6 added another network activity visualization: show strong query activity as radiation around peer orbiter 2010-07-28 11:40:58 +00:00
  • 989948e1a9 fixed generic image parser orbiter 2010-07-27 07:13:15 +00:00
  • e1015ead2c static access to constants orbiter 2010-07-27 06:52:58 +00:00
  • 27d8a8b53e removed wrong com.sun.codec class access in generic image parser orbiter 2010-07-27 06:49:09 +00:00
  • bbf887d879 added generics to UPnP classes orbiter 2010-07-27 06:48:01 +00:00
  • 15e8c13526 ... migrating to HttpComponents-Client-4.x ... (gzip decompression, httploader, robots, ...) sixcooler 2010-07-27 01:16:26 +00:00
  • a55af783bf healing for color blindness orbiter 2010-07-26 22:54:06 +00:00
  • 710e7bb700 de.lng: Added German translation for http://localhost:8080/Ranking_p.html without the actual values (hardcoded in HashMap in Ranking_p.java) mikeworks 2010-07-26 20:57:54 +00:00