Mirrors/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2025-11-05 23:46:12 -05:00
Code Issues Packages Projects Releases Wiki Activity

Commit Graph

  • 98a03249f3 - optimize selectall-javascript-function (in firefox more then 10 times faster now) f1ori 2008-03-28 16:28:36 +00:00
  • 0241d070bc added concurrency to indexing process: - the methods {parsing, semantic analysis (condensing), structure analysis (web structure)} in the serialized indexing path had been made concurrent. - four BlockingQueues handle concurrency and hand-over of the indexing objects, the last object in the queue is stored into a blockingQueue of maximum size 1 to serialize the process for storage (which uses IO and therefore here should not be deserialized) - a concurrency of (CPUs + 1) is default. Single-CPU users will profil from the change because large files cannot block the indexing process any more. - removed the secondary indexing thread, which is superfluous now. Concurrency is default for all users. orbiter 2008-03-28 11:56:28 +00:00
  • 9fb5d661f2 added my Blacklistengines lulabad 2008-03-28 08:18:21 +00:00
  • bca87f1e38 - refactoring of serverThreads: renaming to distinguish busy-threads and blocking-threads - added blockingThreads which are threads that are not driven by pause times but by BlockingQueue lookup orbiter 2008-03-27 12:03:16 +00:00
  • 968c775025 - preparation of parsing/indexing queue for concurrent execution - remote crawl receipts are now transmitted concurrently in separate threads (makes remove crawls much faster!) orbiter 2008-03-26 22:43:38 +00:00
  • 9b0e20fb06 next refactoring step in document indexing to prepare concurrency environment for document parsing orbiter 2008-03-26 19:51:05 +00:00
  • 7f9f639d20 - refactoring and abstraction of index reference (urls) handling: blacklisting is part of reference filtering - refactoring of word/phrase handling: word abstraction from condenser becomes part of index element handling - removed unused code parts from condenser orbiter 2008-03-26 15:37:49 +00:00
  • d6050b9ffb - separated the LURL data storage and Crawl result stack for process supervision. this is another step to enable multiple, concurrent fulltext-indexes - another try to make the yacy-httpc more stable orbiter 2008-03-26 14:13:05 +00:00
  • 8e889de50b *) Added Lotus' patch (http://forum.yacy-websuche.de/viewtopic.php?t=979), user will be taken back to last opened page after making changes in Advanced Settings. low012 2008-03-25 20:40:54 +00:00
  • 8d6a13bc07 refactoring of parsing-condensing-indexing process: - separated parts - removed storagePeer function next step will be parallelization of processes orbiter 2008-03-24 22:51:26 +00:00
  • d3b06913ec protection against seed-db failure during enumeration orbiter 2008-03-23 23:47:41 +00:00
  • 5aa96dbc36 fix for shutdown configuration orbiter 2008-03-23 13:14:57 +00:00
  • 93633abed8 - removed some debugging code from search process - should speed up now - added some profiling code to search event - more time details in PerformanceSearch_p.html orbiter 2008-03-23 00:55:04 +00:00
  • fba46c51d7 fixed non-termination bug in qsort orbiter 2008-03-22 23:15:28 +00:00
  • 541b817502 refactoring of switchboard queueing orbiter 2008-03-22 01:28:37 +00:00
  • fc94fbe224 another improvement to the collection sorting orbiter 2008-03-20 23:11:04 +00:00
  • 11270d450e better quicksort-pivot computation: 30% faster (measured with test program) orbiter 2008-03-20 22:01:12 +00:00
  • 3e44293f07 - fixed a problem with thread pools in row collection - added a line-viewing feature in threaddump orbiter 2008-03-20 14:21:58 +00:00
  • e43051b125 - fixed Threaddump output (html-escaped ie. <init>) - in EcoFS converted comments to javadoc danielr 2008-03-20 10:20:55 +00:00
  • 433ff855f7 - fixed another concurrency problem in collection sorting - fixed a typing problem that was introduced in svn 4579 and caused the crawl monitor to fail orbiter 2008-03-19 23:47:24 +00:00
  • 19286fa2d1 tried to fix seed2.old.db-problem orbiter 2008-03-19 22:35:19 +00:00
  • 5fde618337 changed display of y-marks apfelmaennchen 2008-03-19 19:54:35 +00:00
  • f3996e63b8 tried to fix more deadlocks: - changed connection modes in ftpc - replaced sort tread pool in row collections by new one using util.concurrent. the old pool had caused blockings orbiter 2008-03-19 11:23:43 +00:00
  • 26155cbb5e update daburna 2008-03-18 17:53:54 +00:00
  • 1618bd5505 added submenu daburna 2008-03-17 14:51:04 +00:00
  • 7008a218b3 avoid ConcurrentModificationException in plasmaCrawlerQueues danielr 2008-03-17 13:51:56 +00:00
  • 9d0af17c5b removed debug from startYACY.sh *ups* danielr 2008-03-17 13:02:39 +00:00
  • bf3e5807b0 fixed typo and remove redundant code from startYACY.sh danielr 2008-03-17 12:51:46 +00:00
  • ea59dbf4c2 fixed build problem see http://forum.yacy-websuche.de/viewtopic.php?f=6&t=956&hilit= lulabad 2008-03-17 06:53:20 +00:00
  • 7150b463ff changed handling of default values and database paths: - the default files yacy.init and for the network definition is now moved to the path defaults - the httpProxy.conf is renamed to yacy.conf - the DATA/INDEX/PUBLIC is renamed to the actual network nickname, which should be freeworld or sciencenet more menu entries - added apfelmaennchens alternative search page to the menu - added the new thread dump page to the server log menu point as submenu modifications - modified the thread dump page: sorting by thread type orbiter 2008-03-16 22:31:54 +00:00
  • 54cb097ea4 added .trigger("update") after paging apfelmaennchen 2008-03-16 22:05:50 +00:00
  • 82f17ccee2 just an example sidebar apfelmaennchen 2008-03-16 21:44:51 +00:00
  • 3c710f22cd added server side driven pagination for search tabs apfelmaennchen 2008-03-16 21:44:20 +00:00
  • 25f5035f23 typo lulabad 2008-03-16 15:13:12 +00:00
  • 7fd094fcbe small bug in ftpc: did cot compile in Java 1.5 Please set compiler to Java 1.5-compliance orbiter 2008-03-16 13:41:49 +00:00
  • 0a593a4e4a small changes to Threaddump Servlet lulabad 2008-03-16 13:15:23 +00:00
  • 995b5a2b27 added Threaddump Servlet, see http://forum.yacy-websuche.de/viewtopic.php?f=9&t=862 lulabad 2008-03-16 09:03:52 +00:00
  • f51bad8ae5 FTP: - report connection status (to break if no connection possible) - fixed isFolder() - additional error output - fixed paths with encoded symbols (ie. a%20file.txt) - refactoring danielr 2008-03-15 21:57:55 +00:00
  • 820641938e ftpc: fixed date parsing, some refactoring danielr 2008-03-15 10:56:47 +00:00
  • 4c584dff87 disabled soLinger to prevent that too many connections stay open (it's a TEST!) orbiter 2008-03-15 10:46:55 +00:00
  • d1b9f50b94 added "delete Release" to Update Page lulabad 2008-03-15 10:37:20 +00:00
  • 3d122c125c added "current installed release" to Update Page lulabad 2008-03-15 09:22:07 +00:00
  • 9c989fe5f7 fixed deadlock orbiter 2008-03-15 00:49:16 +00:00
  • c565906050 FTP: - added maxFileSize-check - added timeout for download - fixed dirlist (when all filenames have spaces, change to absolute links) - enhanced isFolder() - make sure data connection is closed, so a new can be opened - refactoring danielr 2008-03-14 16:28:27 +00:00
  • 368b8735b5 added 'close tab' function apfelmaennchen 2008-03-14 15:59:56 +00:00
  • 1a7870df0d FTP: source cleanup (added finals, indention for easier diffs) danielr 2008-03-14 12:35:53 +00:00
  • fa1090113d - next try to fix the networking problem: set the maximum transfer size to less than MTU=1500-52: buffer size <= 1448 - some refactoring of transfer methods (naming) orbiter 2008-03-14 00:16:04 +00:00
  • f63bd26268 fixed search performance / dynamic display of results apfelmaennchen 2008-03-12 21:24:53 +00:00
  • d87d295c68 one more try to fix the connection problem orbiter 2008-03-12 13:13:11 +00:00
  • a3dadcd89b preventing that peer which return a false search result are disconnected orbiter 2008-03-12 00:56:18 +00:00
  • ba622bb240 addendum to svn 4553 orbiter 2008-03-12 00:24:20 +00:00
  • 5530b8e1ca reverted changes to yacy protocol classes: they caused the sciencenet to loose connections a comparisment with the main release 0.57 had been made: this showed a stable network This is an emergency operation to ensure availability of the sciencenet network. orbiter 2008-03-12 00:05:18 +00:00
  • b664a53553 fix for NPE during search orbiter 2008-03-11 15:30:26 +00:00
  • b3d4571a5d fix for bad opensearch file format did you know that yacy has a built-in rss feed reader? try: http://localhost:8080/FeedReader_p.html?url=http://www.tagesthemen.de/xml/rss2 it can be used to fetch search results from other yacy peers: http://localhost:8080/FeedReader_p.html?url=http://sciencenet.fzk.de:8080/yacysearch.rss?search=yacy orbiter 2008-03-11 12:45:27 +00:00
  • b4ed937f1e - modified zone navigation (does still not work correctly) - added dht switch in network definition - 0.574 orbiter 2008-03-11 11:09:38 +00:00
  • 8d0470a5c6 new method to compute search history IDs orbiter 2008-03-10 23:40:56 +00:00
  • 65785da8f2 new method for best hash computation orbiter 2008-03-10 23:28:05 +00:00
  • c75fa90206 adjusted display of search results apfelmaennchen 2008-03-10 23:23:04 +00:00
  • 7a902424af adjusted display of search results apfelmaennchen 2008-03-10 23:22:34 +00:00
  • e0c481decb not class file in SVN .. I guess it is a mistake orbiter 2008-03-10 19:05:06 +00:00
  • 8c15fbf7d1 simple and fast fix for LogStatistics_p lulabad 2008-03-10 18:52:01 +00:00
  • 9eddc1506b - one try to fix the httpd problem - fix for handling of collection index that appears when removing elements - added another navigation method (stub, not working yet) orbiter 2008-03-09 23:58:22 +00:00
  • 7cc4ff05c9 some code enhancements and bugfixes orbiter 2008-03-09 23:48:24 +00:00
  • b4b370586a fixed the box headings apfelmaennchen 2008-03-09 17:25:09 +00:00
  • f7a0804e83 small optical change for the sidebar apfelmaennchen 2008-03-09 13:04:34 +00:00
  • c5f378c7a4 additional images apfelmaennchen 2008-03-09 12:56:11 +00:00
  • 6ebc9b7325 additional images apfelmaennchen 2008-03-09 12:55:26 +00:00
  • 3c686e4e0e for testing puposes - new user interface based on jQuery and Ajax apfelmaennchen 2008-03-09 12:54:37 +00:00
  • f238478cc3 for testing puposes - new user interface based on jQuery and Ajax apfelmaennchen 2008-03-09 12:54:13 +00:00
  • cb8625ca67 for testing puposes - new user interface based on jQuery and Ajax apfelmaennchen 2008-03-09 12:53:50 +00:00
  • 2b43ea9f9d for testing puposes - new user interface based on jQuery and Ajax apfelmaennchen 2008-03-09 12:53:00 +00:00
  • 12cac31be8 for testing puposes - new user interface based on jQuery and Ajax apfelmaennchen 2008-03-09 12:52:42 +00:00
  • 94280b0a39 temporary check-in for testing puposes - new user interface based on jQuery and Ajax apfelmaennchen 2008-03-09 12:51:13 +00:00
  • 6788f8f7c1 fixed error 'FTPC cannot change directory' danielr 2008-03-06 11:59:23 +00:00
  • 7ce76c8ff8 added missing file orbiter 2008-03-05 22:57:53 +00:00
  • bfed9c2da6 - some refactoring in search process - separated sidebars in new search interface and placed them in their own files which can be put in into the search page like plug-ins orbiter 2008-03-05 21:46:55 +00:00
  • a8f54dd244 added new update loacation lulabad 2008-03-05 18:56:01 +00:00
  • 5fd990cc84 sorry, bad commit! orbiter 2008-03-05 13:45:35 +00:00
  • 3445b1e10b *better logging borg-0300 2008-03-05 13:41:54 +00:00
  • 4b0339fec0 *fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=927 *remove some cast *Properties added borg-0300 2008-03-05 13:29:42 +00:00
  • 275a226cc5 refactoring orbiter 2008-03-04 22:45:45 +00:00
  • bc3d3b4c97 fixed rebuildTags() to correctly rebuild folders... apfelmaennchen 2008-03-03 22:36:27 +00:00
  • fbe335db73 consistent use of de.anomic.server.serverMemory to get information about memory statistics danielr 2008-03-02 15:42:50 +00:00
  • ce8adf9907 *) *NIX start script displays the right port number now instead of always displaying 8080 no matter what the port is set to low012 2008-03-01 12:08:24 +00:00
  • 8c06436c4a removing the error-db upon each time a start-up is made. This is necessary because the table uses a lot of RAM and the content is never re-used after Start-Up. orbiter 2008-03-01 09:44:33 +00:00
  • 4fdf695064 - fixed a bug in remote search that prevented that any results had been generated (!) - added a great number of printStackTrace and new exceptions that shall be used to find the cause for a bug in yacy client-server communication which causes the interruption of data transfer which then causes the parser bug for the seed strings. - tried to fix the communication bug on server-side (copy functions) Be aware that the log may be full of errors and bugs - there should not be more bugs but there is more to see orbiter 2008-02-27 23:12:43 +00:00
  • 0ddbed9451 Less memory consumption at start borg-0300 2008-02-27 20:09:22 +00:00
  • 1dce2f1079 more multithreading support: - replaced some synchronized classes by classes from util.concurrent - used a util.concurrent.SynchronousQueue to implement a persistent sorting thread in the very basic kelondroRowCollection which supports sorting with a second thread in case that a double-core processing CPU is used orbiter 2008-02-27 15:16:47 +00:00
  • 6779b455d7 another fix for the punycode parser/generator (should work now!) orbiter 2008-02-26 23:00:20 +00:00
  • 1b127406d0 update to punycode encoding (still not working) orbiter 2008-02-26 22:37:23 +00:00
  • 83860507c9 - added punycode class from gnu idn library - added parser for international domains in yacyURL orbiter 2008-02-26 22:18:40 +00:00
  • 3f2b18a4e7 fix for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=905&hilit=&p=5979#p5979 orbiter 2008-02-26 21:17:41 +00:00
  • 096d0f7240 - better look for Indexcleaner - corrected commas daburna 2008-02-26 15:32:25 +00:00
  • 253a453413 removed possible synchronization deadlock orbiter 2008-02-26 14:05:43 +00:00
  • 3f321ece7d added a search history to the new search page the history distinguishes between different users and identifies them by their ip a history is only shown to the user who submitted the search orbiter 2008-02-25 21:26:49 +00:00
  • c48e25d784 - fixed selection box for topwords - fixed parser detail in condenser orbiter 2008-02-25 19:00:11 +00:00
  • bd2d9f75ae introduced search navigation column on new search page orbiter 2008-02-25 18:19:22 +00:00
  • 87a8747ce3 - enhanced recognition, parsing, management and double-occurrence-handling of image tags - enhanced text parser (condenser): found and eliminated bad code parts; increase of speed - added handling of image preview using the image cache from HTCACHE - some other minor changes orbiter 2008-02-25 14:08:15 +00:00
  • fcc919964b update daburna 2008-02-24 21:18:04 +00:00
  • 652086159a *) Replaced System.err.println() by logging function. Left System.err.println()s as comments to be able to quickly revert changes since gzip is an application with it's own main method and Orbiter maybe wants to keep it this way. low012 2008-02-24 19:56:19 +00:00