Mirrors/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2025-07-19 08:44:42 -04:00
Code Issues Packages Projects Releases Wiki Activity

Commit Graph

  • aa6c32d753 enhanced UTCDiffString orbiter 2011-06-19 12:38:06 +00:00
  • 07cbb6cb5f display cache hit/miss values in correct column lotus 2011-06-15 18:57:04 +00:00
  • f87865a50b always shutdown log, fixes zombie processes in init stop script f1ori 2011-06-15 09:14:51 +00:00
  • 115abc8917 - more attributes for search progress bar - moved cache strategy to cora package orbiter 2011-06-13 21:44:03 +00:00
  • ccad615f58 The Java-XMS and Xmx values for the target of "run" (run YaCy) inserted. suessthomas 2011-06-11 21:22:08 +00:00
  • 7bfa6bb4b6 prevent getting a yacySeed from zero-length-hash-string by chance (for eg.: proxy-crawls got displayed as initiated by some other peer) sixcooler 2011-06-05 22:58:17 +00:00
  • bce280a308 update on options for interface graphics orbiter 2011-06-05 22:48:21 +00:00
  • 77fe69395d added jempbox-1.5.0.jar which is required by pdfbox-1.5 as stated in http://pdfbox.apache.org/dependencies.html orbiter 2011-06-05 20:04:41 +00:00
  • 72a3cd5832 equalize lock icon for Status.html lotus 2011-06-04 18:55:09 +00:00
  • df1725ef43 re-enable POST over proxy, which didn't work since update to httpcore-4.1.1 sixcooler 2011-06-04 13:25:03 +00:00
  • 66c477129e Creates a new network definition, yacy.networks.metager.unit. The YaCy freeworld network used in this network definition, minor enhancements for the feed of MetaGer were integrated. suessthomas 2011-06-03 22:34:42 +00:00
  • 2683162ec5 - added more options to access grid picture, web structure picture and network graphics - remove test class orbiter 2011-06-02 23:27:26 +00:00
  • efcd21e0ed new httpclient, httcore (bugfixrelease) sixcooler 2011-06-02 21:34:50 +00:00
  • d0d6123b18 added a deploy script that can be used to deploy yacy releases into the current release for testing orbiter 2011-06-01 19:52:05 +00:00
  • 265b7ce4f9 removed pause in search test orbiter 2011-06-01 19:49:44 +00:00
  • 0c1b29f3c9 - applied many small performance hacks - added a memory limitation in the zip parser and the pdf parser - added a search throttling: if there are too many search queries are still to be computed, then new requests are not accepted for some time. if after a one second still no space is there to perform another search, the search terminates with no results. this case should only happen in case of DoS-like situations and in case of strong load on a peer like if it is integrated in metager. - added a search cache deletion process that removes search requests in case that throttling happens orbiter 2011-06-01 19:31:56 +00:00
  • 900dacbf97 * improve link rewriting in proxy-url * only rewrites links, which are in current search domain f1ori 2011-06-01 13:27:04 +00:00
  • 7fea51ecee check filter to bee a correct pattern on edit CrawlProfiles see; http://forum.yacy-websuche.de/viewtopic.php?f=5&t=3277&p=22662#p22660 sixcooler 2011-05-31 16:13:33 +00:00
  • af63aa1d0e added fresh links to java regular expression api-doc orbiter 2011-05-31 13:33:04 +00:00
  • dc855d881b * further improve proxyurl f1ori 2011-05-30 21:25:20 +00:00
  • 761b1c71dc added latest pdfbox orbiter 2011-05-30 14:56:36 +00:00
  • a7a6b392f5 code cleanup orbiter 2011-05-30 10:16:43 +00:00
  • fe0c08455b more concurrency (enhancement) hacks orbiter 2011-05-30 08:53:58 +00:00
  • 0e9a99cb05 another resource hack orbiter 2011-05-30 07:51:18 +00:00
  • 535b6b953c more hacks to omit superfluous string object allocation orbiter 2011-05-30 07:31:17 +00:00
  • 87082f407e less String object creation during search orbiter 2011-05-30 04:19:20 +00:00
  • ab5a16b957 lesse memory occupation during ranking and faster host navigator orbiter 2011-05-29 20:33:12 +00:00
  • b8aa41a1b4 show nsis version in installer again for more transparency lotus 2011-05-27 16:30:55 +00:00
  • 1489ebeedf one more hack to free ram for search events orbiter 2011-05-27 14:26:37 +00:00
  • 3c2b994bd6 write access/load time to solr index orbiter 2011-05-27 12:35:08 +00:00
  • a36fda991e hack to increase speed of url hash computation orbiter 2011-05-27 12:34:38 +00:00
  • 752576b521 - localsearch test script does also a snippet-fetch - killYACY.sh does not need a sleep between kill -3 and kill -9 orbiter 2011-05-27 12:08:45 +00:00
  • ddcc333acc * fix negative result counts f1ori 2011-05-27 11:21:00 +00:00
  • fa734bdf9f better memory protection in search logger orbiter 2011-05-27 11:18:22 +00:00
  • dbea40d536 - changed snippet fetch strategy logic: do not check if entry is in cache. This should reduce IO load on the HTCACHE which is a showstopper during large number of search requests - forced a possible short memory status when a search is started to flush caches that may cause search-heaps with resource contention effects orbiter 2011-05-27 09:32:03 +00:00
  • 4bea3f9714 hack to reduce resource contention caused by massive UTF8 decodings which use java.nio resources: used a ASCII String <-> byte[] conversion wherever possible. Many Strings in YaCy are hashes which are pure ASCII (base64 hashes). The new ASCII String <-> byte[] conversion method have less computation overhead than the UTF8 conversion. orbiter 2011-05-27 08:24:54 +00:00
  • 746e3c3b06 Replaced a widely-used Property Object in the httpd with HashMap<String, Object> which is not synchronized like Properties A synchronization is not needed here and applies an overhead to the httpd process which is now removed. orbiter 2011-05-26 16:34:35 +00:00
  • cc239b18cd fix for IPv6 localhost proxy client orbiter 2011-05-26 16:24:11 +00:00
  • fcb7525047 * add .gitignore to svn, so it doesn't get lost on git syncronization f1ori 2011-05-26 16:13:07 +00:00
  • 14e1666b21 * fix replacing regexes in url proxy f1ori 2011-05-26 16:09:29 +00:00
  • e28bd0d038 fix for some possible causes of memory leaks orbiter 2011-05-26 14:35:32 +00:00
  • 09ba6814c0 - non-blocking word hash computation with dynamic digest object generation (this was important!) - (very) small performance enhancement in did-you-mean orbiter 2011-05-26 12:58:11 +00:00
  • 8d9b5dda3b disabled did-you-mean computation for json and rss search results where this info is not used orbiter 2011-05-26 12:35:24 +00:00
  • 10e2f588f8 - enhanced ybr ranking computation - many speed/performance hacks - added solr charding and new charding web interface - added option to switch off the yacy index when using solr - added new fail-url categories which are used to make a distinction which fail-urls to be sent to solr - refactoring/renaming of some method names to distinguish host/url hashes better - a large number of bug/npe fixes orbiter 2011-05-26 10:57:02 +00:00
  • bd55dcee50 - commented out experimental distributed ranking loading - less threads for blocking threads - disable all threads for DHT transmission for networks with zero peers orbiter 2011-05-24 21:08:01 +00:00
  • 98c4d25185 fix for endless loop in FTP crawling, see http://bugs.yacy.net/view.php?id=32 orbiter 2011-05-24 10:06:20 +00:00
  • d1dbbd956a always use a template method cache even if the template cache flag is set to false. This flag is only used to make dynamic updates to the template files, to not dynamic updates to the rewrite methods (which is not possible without recompiling). low memory usage is guaranteed by the usage of soft references which are dropped before an OOM is thrown orbiter 2011-05-24 09:31:07 +00:00
  • 0d040ff6bb fix for bug 0000036: no crawling of https pages orbiter 2011-05-24 09:14:32 +00:00
  • 3ed4a09368 small features, some bug fixes and performance hacks orbiter 2011-05-23 21:08:04 +00:00
  • e55c254f7b enhanced logging orbiter 2011-05-22 20:12:13 +00:00
  • 3ec94d87c4 show dom counter only for active crawls where the dom counter is enabled within the crawl profile orbiter 2011-05-22 19:34:20 +00:00
  • e3ee43e6ed these YBR files are not needed any more orbiter 2011-05-18 14:27:24 +00:00
  • b45701d20f this is a re-implementation of the YaCy Block Rank feature This time it works like this: - each peer provides its ranking information using the yacy/idx.json servlet - peers with more than 1 GB ram will load this information from all other peers, combine that into one ranking table and store it locally. This happens during the start-up of the peer concurrently. The new generated file with the ranking information is at DATA/INDEX/<network>/QUEUES/hostIndex.blob - this index is then computed to generate a new fresh ranking table. Peers which can calculate their own ranking table will do that every start-up to get latest feature updates until the feature is stable - I computed new ranking tables as part of the distribition and commit it here also - the YBR feature must be enabled manually by setting the YBR value in the ranking servlet to level 15. A default configuration for that is also in the commit but it does not affect your current installation only fresh peers - a recursive block rank refinement is implemented but disabled at this point. it needs more testing orbiter 2011-05-18 14:26:28 +00:00
  • d27a0a67ff fix in log initialization according to hint from Dominic orbiter 2011-05-17 15:53:59 +00:00
  • 205cc75157 abstraction of surrogate main element (xmlns:geo was missing for wiki extracts) orbiter 2011-05-17 08:57:49 +00:00
  • 021840e5ba removed (almost) deadlocks and unnecessary CPU load orbiter 2011-05-17 00:00:01 +00:00
  • 3d879e0995 test file not needed orbiter 2011-05-16 23:19:11 +00:00
  • 123375bfba added a new yacy protocol servlet 'idx'. This returns an index to one of the data entities that is stored in YaCy. This servlet currently only serves for indexes to the web structure hosts. It can be tested by calling http://localhost:8090/yacy/idx.json?object=host This yacy protocol servlet is the first one that returns JSON code and that also shows index entries in a readable format. This will make the development of API applications much easier. This is also an example implementation for possible json versions of the other existing YaCy protocol interfaces. orbiter 2011-05-15 22:57:31 +00:00
  • d326f1486a added timeout setting to scanner interface orbiter 2011-05-14 11:30:41 +00:00
  • f0d5ddfa92 *) preventing potential NPE which occured if user deleted DATA/RELEASE manually and opened ConfigureUpdate_p.java then low012 2011-05-14 09:23:19 +00:00
  • 5c981762c6 added bigrange option for network scan orbiter 2011-05-14 09:13:16 +00:00
  • c55787d07c *) revert of r7667 low012 2011-05-14 09:03:18 +00:00
  • bade61696f speed-up of network port scanner orbiter 2011-05-14 09:03:16 +00:00
  • b04382bc59 added topmenu as defined for search to wiki orbiter 2011-05-14 08:29:16 +00:00
  • 229df8b626 restart link after memory changed lotus 2011-05-13 17:24:03 +00:00
  • 1d8b0f74f4 one more fix for SVN 7713 orbiter 2011-05-13 15:31:24 +00:00
  • 0960261769 fix for svn 7713 orbiter 2011-05-13 15:20:57 +00:00
  • 7e368000c8 transparent progress bar apfelmaennchen 2011-05-13 13:40:23 +00:00
  • 5b579e21a3 code cleanup orbiter 2011-05-13 06:21:40 +00:00
  • fcd4b03892 show progress of search after display of results is finished orbiter 2011-05-13 06:20:00 +00:00
  • 8b63d7637d revert 7710, lotus 2011-05-09 20:06:05 +00:00
  • 965aac5ebb * proxy works almost Florian Richter 2011-05-09 18:20:34 +02:00
  • 440e3ba887 Windows Installer: - remove firewall-handling for WinXP (can only open for JRE not for special port) - Vista/Win 7: open port 1900 for communication with router (uPnP) pca 2011-05-09 06:16:35 +00:00
  • 039126cfaf better handling of on/off switched solr indexing orbiter 2011-05-08 22:47:20 +00:00
  • dc54915df4 fix for very bad compare orbiter 2011-05-08 08:45:58 +00:00
  • f123dbec79 fix in heuristics config lotus 2011-05-07 18:52:20 +00:00
  • 897b4e8b9c another hack to prevent black images orbiter 2011-05-07 07:45:02 +00:00
  • 9248a4eef4 reduce teh effect of 'Bildersuche findet generierte HTML-Seiten als Bilder' see http://bugs.yacy.net/view.php?id=9 orbiter 2011-05-07 07:37:46 +00:00
  • 0621a15f89 fix for wrong search result counter: added a counter for all filtered out entities see also http://bugs.yacy.net/view.php?id=5 orbiter 2011-05-06 23:04:27 +00:00
  • 61c9a791c4 YMarks: sidebar with tabs for tags and folders apfelmaennchen 2011-05-06 21:36:35 +00:00
  • 9c33b2fb58 fix for String Matcher in case that no snippet is returned (NPE) orbiter 2011-05-05 23:11:03 +00:00
  • 76f2817e00 a fix for the snippet computation and hopefully better snippets orbiter 2011-05-05 23:05:38 +00:00
  • deda54d684 - relaxed matching of string-search (this is now case-insensitive) - added transport of string-search pattern to remote search protocol - fixed a problem parsing snippets with a '-' inside orbiter 2011-05-05 22:37:06 +00:00
  • 8fd4e8ea98 proper jre version (without -s in filename) lotus 2011-05-05 20:03:27 +00:00
  • 15e3a57b4e removed unused functions in condenser orbiter 2011-05-05 09:23:10 +00:00
  • 6e42d4de88 - added full-String search function: find things that match exactly what is quoted in the query - re-structuring authentification methods to fix a problem with API steering orbiter 2011-05-05 00:25:14 +00:00
  • 8e10b82280 small fix for solr export orbiter 2011-05-03 22:21:45 +00:00
  • 8b8db2aaba YMarks: some small changes/fixes apfelmaennchen 2011-05-03 21:21:06 +00:00
  • 441035f1f4 YMarks: some improvements to flexigrid quick search on YMarks.html apfelmaennchen 2011-05-02 20:11:58 +00:00
  • 6fa439c82b - refactoring of robots - added option to crawler to send error-URLs to solr - changed solr scheme slightly (no multi-value fields where no multi values are) orbiter 2011-05-02 14:05:51 +00:00
  • 1ea0bc775c @apfelmaenchen: is this the expected, but forgotten change? Please correct if I'm wrong (this let me build Yacy again) sixcooler 2011-05-02 10:46:05 +00:00
  • e7c2ea193b YMark: - general improvements on importers, especially on auto tagging - added get_tags (needed for tag clouds etc.) - improved flexigrid support - added YMarks.html (not fully working) that will eventually replace Bookmarks.html apfelmaennchen 2011-05-01 21:42:48 +00:00
  • e3d19d0a90 fix in Document inboundlinks/outboundlinks sorting orbiter 2011-05-01 15:49:04 +00:00
  • 5e2d38ef19 Windows Installer: - fix for firewall Vista/Win7 - update to JRE 1.6 u25 - TODO: fix for firewall WinXP and setting for uPnP (Port 1900) pca 2011-04-30 19:32:07 +00:00
  • 4e8fa03514 added more attributes to html evaluation orbiter 2011-04-29 15:36:44 +00:00
  • 3b578a28ef some patches to prevent that empty or bad IP information is broadcasted - on client-side: fix bad IP reports from remote Peers by replacing their reported IP with their server IP if the reported IP is bad, broken or disallowed - on server-side: the same during a peer ping (here the ping'ed server acts also as client during the back-ping) and also when receiving a message or a search where the client sends also its seed. Here the IP is replaced by the client IP if the reported IP is broken or bad orbiter 2011-04-29 10:58:12 +00:00
  • 361841df16 another patch according to http://bugs.yacy.net/view.php?id=26#c36 orbiter 2011-04-29 02:26:50 +00:00
  • 37fede9d30 better logic for proper seed ip recognition and better error messages orbiter 2011-04-29 02:19:13 +00:00
  • 8b95a26866 better magic orbiter 2011-04-29 02:00:37 +00:00
  • 2700a58e5a added a magic to the peer ping that will be used in case that the contacting peer requests that it's reported IP shall be used for a back-ping. The back-ping now also returns the same magic which will make it possible that the requested peer can verify that the back-pinged peer is actually the same peer. orbiter 2011-04-29 01:52:20 +00:00