Mirrors/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2025-11-05 23:46:12 -05:00
Code Issues Packages Projects Releases Wiki Activity

Commit Graph

  • a52681dd49 add buffering for the performance graph to avoid ConcurrentModificationException closes: http://forum.yacy-websuche.de/viewtopic.php?f=6&t=628 fuchsi 2007-12-18 15:59:35 +00:00
  • 814aff60bd - (re-)activated ftp protocol. see discussion here: http://forum.yacy-websuche.de/viewtopic.php?f=6&t=623&hilit=&p=3875#p3875 - set default-flushsize for pro to 500 orbiter 2007-12-18 00:14:44 +00:00
  • 6fbda9ef4f *) cleaned up code *) added new filter 'FILTER_INVERT' and new method ymageMatrix.invert() to use it (does not work where characters have been written with ymageToolPrint.print(), haven't found the reason yet) *) fixed a possible arrayOutOfBoundsExceptions in filter() if y-value of area to use filter in was larger than height of image filter is used with low012 2007-12-16 22:15:27 +00:00
  • b0aefcf2f3 #small update daburna 2007-12-16 15:23:33 +00:00
  • 21f7e13fa1 fix stupid tiny bug introduced in rev 4276 that broke request URL parsing almost completely fuchsi 2007-12-16 00:33:32 +00:00
  • 5d406d0094 - fixed url "file extension" parsing when there is no extension (like http://yacy.net/ would have extracted .net/) - removed unecessary code + minor cleanup fuchsi 2007-12-14 20:03:26 +00:00
  • 21b8d1b918 small cosmetic change for static fields in serverCore (special protocol ASCII entities) to improve readability fuchsi 2007-12-14 19:17:54 +00:00
  • 270d016d89 fix for missing anonymization in search profiling orbiter 2007-12-12 18:57:43 +00:00
  • e3e4f06be4 enhanced search result preparation in the case that no result is found (fast abandon of search) orbiter 2007-12-12 14:18:42 +00:00
  • 321553afc2 #updated German language file daburna 2007-12-12 12:15:37 +00:00
  • 59e83b2fb6 #updated french language file by translation out of the wiki. made by Mikayé daburna 2007-12-12 11:39:09 +00:00
  • 1bd02762de Improve HTTP/ICAP header processing. - workaround for illegal line endings (LF only), closes: http://forum.yacy-websuche.de/viewtopic.php?f=6&t=595 - fixed bug where we didn't break the processing immediately on EOS (the loop was run until the buffer was completely filled with -1) - further performance improvements (one simple loop, avoid double processing of every byte and unnecessary temporary buffers) fuchsi 2007-12-12 06:37:18 +00:00
  • 01554f4012 fixed bug with double-check in crawler orbiter 2007-12-12 01:32:25 +00:00
  • b1e08d354c repaired indexing after search snippet loading orbiter 2007-12-12 00:33:26 +00:00
  • 48138952ff added memory measurement for index recreation to avoid OOM during index RAM space extension orbiter 2007-12-11 15:07:03 +00:00
  • 5a80359b0e new default remote favicon for search results orbiter 2007-12-11 12:40:48 +00:00
  • 9e23acf2d6 introduced new 'authority' ranking property orbiter 2007-12-11 01:32:58 +00:00
  • a1b80017e0 fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=594&p=3630#p3630 orbiter 2007-12-10 15:00:01 +00:00
  • a3bfd668aa opening of array files at startup time, not when first time the web index is accessed this speeds up the first search after startup orbiter 2007-12-10 02:40:16 +00:00
  • ca488e03f5 fixed authorization case orbiter 2007-12-10 02:04:48 +00:00
  • 6a3a292015 - smoothed ymage font - changed position of status banner orbiter 2007-12-10 01:47:04 +00:00
  • 7397152e04 *) quick hack for antialiasing, works only on borders now => less blurry image *) code is not finished, needs refactoring, still thinking about how to do it best low012 2007-12-09 19:17:24 +00:00
  • 4331e52d1c fixed too small peer number in banner orbiter 2007-12-08 02:48:19 +00:00
  • 515e1bde6d - fixed bug with constraint default - 0.556 - default RAM for pro releases now 120MB (because pro will become default) orbiter 2007-12-08 02:04:50 +00:00
  • 2954f96fae - removed public peer info box on status page, this info can now be seen in the status banner - added peer count to banner - added some values to protected status box orbiter 2007-12-08 01:39:59 +00:00
  • 4eb40c4f61 *) added 2 filters: blur and antialiasing (which in fact is nothing more than a mild blur) to ymageMatrix *) antialiasing is used for logo in banner low012 2007-12-07 22:51:13 +00:00
  • aeb1cf83a6 - corrected banner link (relative now) - changed color mode (replace) for banner - changed default color (fits to default skin) of banner in status orbiter 2007-12-07 21:25:36 +00:00
  • 5185acaf41 - reduced default search time - this can be configured using the network definition file orbiter 2007-12-07 03:25:57 +00:00
  • e22014dc83 some memory enhancements when generating and displaying ymage objects orbiter 2007-12-07 02:15:12 +00:00
  • f243e338cf implemented online caution also for local and remote search orbiter 2007-12-06 21:53:17 +00:00
  • 6680634703 removed unnecessary functions orbiter 2007-12-06 21:29:47 +00:00
  • 3ff09ad6b4 #updated french language file by translation out of the wiki. made by Marsupoil daburna 2007-12-06 18:20:46 +00:00
  • c57eb76b13 removed CMY color model from ymage classes and re-introduced RGB color model orbiter 2007-12-06 01:06:17 +00:00
  • b46bcaa5d8 changed method of profiling orbiter 2007-12-04 20:19:13 +00:00
  • 76cd6ed6f6 *) New methods to insert bitmaps that feature transparencies. *) Logo background is transparent now. (Using pixel at (0,0) to determine which color is transparent. Too dirty?) *) Logo is loaded through filesystem instead of HTTP now. low012 2007-12-04 19:45:50 +00:00
  • be214e594f - generalized ymage initialization options - auto-adoption of performance memory graph to needed dimension orbiter 2007-12-03 02:35:28 +00:00
  • ee8a177c26 *) Logo is in the middle of free space now. *) Fixed bugs in insertBitmap() low012 2007-12-02 21:20:11 +00:00
  • 72698fcd36 *) Banner features a logo now. It does not look nice, but at least it works. Banner is not finished yet. low012 2007-12-02 20:37:12 +00:00
  • 39d0f10ca1 Fix parsing oof dates in HTTP headers. RFC 2616 requires a client to support RFC 1123 (default), RFC 1036 and ANSI C formatted date strings (we only supported 1123 before). fuchsi 2007-11-30 20:47:27 +00:00
  • 3f848e282b PerformanceMemory_p now does a automatic update of memory graph every second orbiter 2007-11-30 11:28:26 +00:00
  • aefb3f7765 added memory graph picture to PerformanceMemory_p.html orbiter 2007-11-30 03:22:42 +00:00
  • ea81d97cfc fix for bad full domain crawl depth adoption (maximum depth is 8, because higher depth will cause that remote crawls do not work) orbiter 2007-11-30 00:37:58 +00:00
  • f645408ae9 added url retrieve option to uls.xml interface orbiter 2007-11-29 22:12:01 +00:00
  • cc20870267 fix for constraint handover problem: old yacy versions set a catchall-constraint if no constraint is given, but the new versions expect a null-constraint. see http://forum.yacy-websuche.de/viewtopic.php?f=5&t=565&hilit= orbiter 2007-11-29 21:03:47 +00:00
  • 5ba415570e #updated German language file now the new IndexControlsites are complete translated daburna 2007-11-29 17:14:04 +00:00
  • 9b0ae4b989 added referrer to remote crawl url list orbiter 2007-11-29 13:58:00 +00:00
  • 18e516317d Fix problem with buggy HTTP-Servers which send illegal control characters in HTTP-Headers, they are ignored now. fuchsi 2007-11-29 06:02:45 +00:00
  • 7d5544e9b1 added some security checks to new remote crawl pull method to prevent that indexer is overloaded orbiter 2007-11-29 02:54:59 +00:00
  • d59c1a7936 removed test data orbiter 2007-11-29 02:11:48 +00:00
  • 89b9b2b02a redesigned remote crawl process: - instead of pushing urls to other peers, the urls are actively pulled by the peer that wants to do a remote crawl - the remote crawl push process had been removed - a process that adds urls from remote peers had been added - the server-side interface for providing 'limit'-urls exists since 0.55 and works with this version - the list-interface had been removed - servlets using the list-interface had been removed (this implementation did not properly manage double-check) - changes in configuration file to support new pull-process - fixed a bug in crawl balancer (status was not saved/closed properly) - the yacy/urls-protocol was extended to support different networks/clusters - many interface-adoptions to new stack counters orbiter 2007-11-29 02:07:37 +00:00
  • 69521d92e5 Add another external dependency from PDFBox package ("Bouncy Castle"). This is necessary for parsing of some encrypted PDF files. bcprov-jdk14-132.jar is the binary jar as it is provided in the PDFBox-0.7.3 package (same as our FontBox, PDFBox packages). fuchsi 2007-11-27 23:13:26 +00:00
  • 90a02990d2 NPE fix, see http://forum.yacy-websuche.de/viewtopic.php?f=6&t=549&hilit=&p=3383#p3383 orbiter 2007-11-23 09:26:35 +00:00
  • 2fcd18a972 - fixed bad behaviour of search event worker processes - fixed export of url lists in xml orbiter 2007-11-23 01:08:16 +00:00
  • 445c0b5333 added domain list extraction and html export format to URL administration menu http://localhost:8080/IndexControlURLs_p.html orbiter 2007-11-22 20:47:06 +00:00
  • d8d77fc4b2 fix for NPE, see http://forum.yacy-websuche.de/viewtopic.php?f=6&t=549&hilit=&p=3368#p3368 orbiter 2007-11-22 18:15:28 +00:00
  • bf6952abe7 - added url export to http://localhost:8080/IndexControlURLs_p.html - removed command-line option to export urls orbiter 2007-11-22 16:52:44 +00:00
  • af10f729df fixed image search and favicon loading orbiter 2007-11-22 01:34:29 +00:00
  • edba2b7bcc fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=543 orbiter 2007-11-21 23:26:51 +00:00
  • c48b73cda2 redesign of ranking data structure - the index administration now uses the same code base for url selection and collection as the search interface. The index administration is therefore a good test environment for ranking order control - removed old postsorting-algorithms, will be replaced with new one - fixed many bugs occurred before during ranking; especially the contraint filtering method removed too many links - fixed media search flags; had been attached to too many urls. The effect should be a better pre-sorting before media load within snippet fetch orbiter 2007-11-21 23:14:57 +00:00
  • 6f1308da2f - some enhancements to IndexControlURLs (shows more links, connects referrer to another query) - some refactoring to search process orbiter 2007-11-17 01:53:02 +00:00
  • bf9a9e4e5e fix for NPE in IndexControlRWIs_p.java orbiter 2007-11-16 16:37:45 +00:00
  • c527969185 - enhanced monitoring of ranking parameters for details, please try http://localhost:8080/IndexControlRWIs_p.html - fixed computation of ranking ordering in some cases orbiter 2007-11-16 14:48:09 +00:00
  • bd5673efbe added cleaning of search event before opening the index administration orbiter 2007-11-15 12:49:13 +00:00
  • 55da871211 preparations for better ranking: better debugging of index properties to do this, the index administration interface was extended. It is now possible to select parts of a index. See properties shown in interface after a word search for details. orbiter 2007-11-15 03:03:18 +00:00
  • 383dc815d2 *) fix for commit 4212 low012 2007-11-14 19:14:53 +00:00
  • 3491531cea - fixed 'appears in url' flag in index generation - extended index administration page, shows some properties to the web links now orbiter 2007-11-14 01:15:28 +00:00
  • 19176e12e2 -corrected typo made in 4213 -updated translation daburna 2007-11-13 10:13:04 +00:00
  • ec7ba0d3d0 - fixed problem with too small sort fields (sortbound was not set) - slightly changed handling of date in indexURLEntry orbiter 2007-11-13 02:24:10 +00:00
  • ca4ca79eba removed wrong hints to installation page. orbiter 2007-11-12 23:45:52 +00:00
  • a01c42575d *) 404 error pages will be displayed with correct CSS and favicon now (http://forum.yacy-websuche.de/viewtopic.php?t=482) low012 2007-11-12 20:18:47 +00:00
  • 2b57d64de3 *) typo (http://forum.yacy-websuche.de/viewtopic.php?t=428) low012 2007-11-12 19:18:51 +00:00
  • bc2368e907 fix for problem with remote crawl referrers orbiter 2007-11-12 16:32:50 +00:00
  • 875096552f fix for NPE in case that remote search results are empty orbiter 2007-11-12 15:54:50 +00:00
  • 64b3b79e44 - fix for termination problem with uniq() - addition to seed dna interpretation orbiter 2007-11-12 14:39:30 +00:00
  • 0abf33ed03 - tried to remove deadlock - enhanced searchtime in kelondroRowSets - enhanced uniq() - reverse enumeration causes less time in case of mass removal of doubles orbiter 2007-11-12 01:14:51 +00:00
  • a4010f7dc8 *) fixed bug where dots were added after numbers < 1000: "123" was transformed to "123." which is undesirable low012 2007-11-11 21:42:50 +00:00
  • da73cde86e #German language file - some cosmetics daburna 2007-11-11 20:32:51 +00:00
  • 2421127612 fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=513&hilit= orbiter 2007-11-11 01:32:54 +00:00
  • 2e91b724ad fix for yacysearch/rss-feed bug orbiter 2007-11-11 00:42:31 +00:00
  • d0d2771883 disabled multiprocessoring of rowCollection.sort for testing purpose orbiter 2007-11-11 00:28:22 +00:00
  • edc4da5317 fix for division by zero in test reoutine orbiter 2007-11-10 08:57:00 +00:00
  • df38aaf7bd update to RowCollection sort speed-enhancements: - better handling of small collections (less overhead) - usage of pre-sorted limits - different re-sort limit - more testing procedures orbiter 2007-11-09 15:34:11 +00:00
  • 0eb60cfe6f better handling of seed properties orbiter 2007-11-09 09:40:42 +00:00
  • ecba35de72 enhanced computing speed of kelondro core function: sorting the enhancement was made by using better organized data structures and multi-threading during the sort. A sort can be divided into two separate processes when the first partition of the quicksort algorithm was done. Generating a separate thread and starting the thread takes only 10 milliseconds, so using a separate thread makes only sense if the data amount is large. statistics about the speed-up: without ehancement: 250 milliseconds for 100000 entries with data structure enhancement: 170 milliseconds for 100000 entries with additional second thread (if second processor is present): 130 milliseconds. orbiter 2007-11-09 00:51:38 +00:00
  • 6eaa5a0e64 enhanced local search speed. The ranking process is now 6 times faster that before. orbiter 2007-11-07 22:38:09 +00:00
  • 425e4ead66 Allow absolute paths in configuration settings. - before absolute paths would be expanded incorrectly, e.g.: fooPath=/a/b/c would become /path/to/yacy/root/a/b/c. Now you can put nearly every dynamically generated data with a configurable path to a location outside of yacys root dir without having to use symlinks (probably good for third party distribution packaging). - abstractServerSwitch.getConfigPath(setting, default) returns a File instance, either with an absolute path or relative to the applications root path. fuchsi 2007-11-04 10:36:25 +00:00
  • e8d32d9f62 other loglevel borg-0300 2007-11-02 16:06:54 +00:00
  • a5d28785b1 less OOM (works for me) borg-0300 2007-11-02 14:55:46 +00:00
  • 794d296129 project link update orbiter 2007-11-01 20:40:15 +00:00
  • ccbfb15b6b enhancement to crawl stacker enqueue order orbiter 2007-11-01 00:57:32 +00:00
  • 93905e5c7b fix for show-more bug orbiter 2007-11-01 00:55:39 +00:00
  • 5c5344ae97 Beautify log hermens 2007-10-31 16:29:07 +00:00
  • 35cf196204 transferRanking(): Do not flush more ranking files than requested by caller. hermens 2007-10-31 15:55:52 +00:00
  • d0aa8cf25d Only update handshaked peer's last seed date if it has not been updated recently. Unil now the newer data was overwritten by old data from before the handshake. hermens 2007-10-31 15:47:48 +00:00
  • 8f9d65da67 Small corrections to dhtFlushControl() - Test wCacheMaxChunk against maxURLinCache(), not getMaxWordCount(). This triggered a flush everytime dhtFlushControl() was called. - If triggered, flush at least 1 entry. hermens 2007-10-31 14:21:58 +00:00
  • 55c87b3b12 changed behavior of crawl stacker - final flush only when tabletype = RAM - prestacker (dns prefetch) only if tabletype = RAM and busytime <= 100 - number of maximun entries in stacker is configurable in yacy.init (stacker.slots) orbiter 2007-10-31 11:32:40 +00:00
  • 18144043e6 Correct UTC Offset at beginning/end of daylight savings time hermens 2007-10-30 19:20:02 +00:00
  • 4fefa53135 removed parser object pool, see also svn 4106 orbiter 2007-10-29 12:14:18 +00:00
  • 35b1bd66cd The (old) yacy web page does not need to part of the yacy distribution. The old yacy home page will be replaced by a new one. orbiter 2007-10-29 11:27:50 +00:00
  • 87b297b4d2 update of link to english forum orbiter 2007-10-29 10:50:27 +00:00