Commit Graph

Select branches

Hide Pull Requests

master

#1

#10

#100

#100

#101

#101

#104

#105

#108

#110

#114

#12

#122

#127

#127

#128

#135

#137

#137

#14

#144

#149

#151

#154

#155

#157

#159

#167

#167

#18

#183

#186

#188

#2

#21

#218

#219

#220

#226

#23

#238

#239

#240

#250

#258

#26

#260

#261

#27

#278

#28

#283

#29

#292

#3

#303

#32

#325

#327

#332

#333

#334

#338

#34

#355

#356

#357

#37

#38

#389

#39

#390

#4

#40

#402

#403

#404

#405

#408

#41

#41

#419

#42

#420

#436

#437

#439

#44

#440

#442

#443

#447

#448

#45

#45

#46

#461

#462

#463

#464

#47

#472

#478

#484

#485

#488

#489

#490

#495

#5

#5

#51

#52

#53

#533

#535

#54

#540

#541

#55

#553

#554

#556

#559

#56

#560

#562

#564

#567

#568

#575

#576

#58

#580

#583

#584

#585

#588

#589

#594

#596

#597

#598

#60

#600

#603

#605

#607

#61

#610

#612

#614

#615

#616

#617

#619

#62

#621

#628

#633

#642

#649

#65

#650

#651

#657

#66

#661

#661

#664

#666

#67

#670

#672

#678

#68

#682

#686

#688

#689

#69

#690

#692

#693

#694

#695

#696

#697

#698

#699

#700

#704

#71

#710

#710

#715

#718

#72

#721

#722

#730

#734

#737

#76

#77

#78

#8

#88

#9

#9

#91

#93

#93

#97

#97

#98

0.99

1.0

Release_1.0

Release_1.01

Release_1.02

Release_1.03

Release_1.04

Release_1.1

Release_1.2

Release_1.3

Release_1.4

Release_1.5

Release_1.6

Release_1.68

Release_1.7

Release_1.72

Release_1.80

Release_1.82

Release_1.90

Release_1.92

Mono Color

• e48ce5d80e - style change for search box: larger font, selected by default - style change for search results: by default no parser, size, image info orbiter 2011-09-14 09:05:06 +00:00
• 5905392ca3 redesign and simplification of main menu; bundling of some sub-menues orbiter 2011-09-14 01:24:56 +00:00
• e5a93a1742 fix for image name orbiter 2011-09-14 00:33:08 +00:00
• 5fd4f3fef8 fresh look for yacy icons orbiter 2011-09-14 00:03:33 +00:00
• 95790b82d9 replaced old-style favicon orbiter 2011-09-13 23:34:36 +00:00
• bb0c045036 fix for problem with relocation of network orbiter 2011-09-13 18:46:11 +00:00
• 85a5487d6d YaCy can now use the solr index to compute text snippets. This makes search result preparation MUCH faster because no document fetching and parsing is necessary any more. orbiter 2011-09-13 14:39:41 +00:00
• 0819e1d397 protection against OOM cases in image parser. See also bugs.yacy.net/view.php?id=54 orbiter 2011-09-09 23:00:45 +00:00
• 52a2b3f110 try to fix bug http://bugs.yacy.net/view.php?id=26 orbiter 2011-09-08 19:13:19 +00:00
• b0b4886618 try to avoid the unresolved pattern in search result orbiter 2011-09-08 18:47:00 +00:00
• 2e8c8ebf54 fix for clear commands orbiter 2011-09-08 12:54:10 +00:00
• 2cba860693 - fix for wrong entries in NOLOAD indexing queue (that caused that urls had been only indexed based on their url and not loaded) - patch for better urls to solr admin interface orbiter 2011-09-08 12:23:55 +00:00
• 2842ce30d6 added synchronization in ReferenceContainer and logging for shrinking orbiter 2011-09-07 22:15:01 +00:00
• cec3836e73 added reference limitation to IndexControlRWIs_p.html servlet orbiter 2011-09-07 21:47:54 +00:00
• ecb4986b38 refactored stuff from last commit to ReferenceContainer see: http://forum.yacy-websuche.de/viewtopic.php?f=5&t=3353&p=23163#p23163 the limiting of references is disabled per default to enable this set yacy.conf - index.maxReferences to a value of e.g. 100000 sixcooler 2011-09-07 18:55:16 +00:00
• f7c4abfdd7 limit references per blob & term to the 100.000 youngest sixcooler 2011-09-07 13:08:06 +00:00
• 28f5b79deb added a fast mass-deletion method orbiter 2011-09-07 11:42:06 +00:00
• a70dbce41c added another file tool class to yacy-cora orbiter 2011-09-07 10:09:35 +00:00
• 49e5ca579f added new configuration property "crawler.embedLinksAsDocuments". If this is switched on (this is default now), the all embedded image, audio and video links from all parsed documents are added to the search index as individual document. This will increase the search index size dramatically but will also enable us to create a much faster image, audio and video search. If the flag is switched on, the index entries are also stored to a solr index, if this is also enabled. orbiter 2011-09-07 10:08:57 +00:00
• e02bfbde56 fix for solr url orbiter 2011-09-07 09:07:40 +00:00
• 41e146116a fixes size of document in case the server doesn't give the size in the header f1ori 2011-09-05 12:21:25 +00:00
• 580beb12a5 reverting SVN 7863; the synchronization was needed and no synchronization causes repeated DNS lookup for the same hosts orbiter 2011-09-05 00:26:27 +00:00
• 44d6416e2d ensure termination of shrink() orbiter 2011-09-05 00:22:21 +00:00
• 52230a6864 replaced catching of Exception with Throwable, which catches also Errors orbiter 2011-09-05 00:09:48 +00:00
• 877eaf6bcb switched off logging of org.apache.http which was suddenly switched on by default (??) orbiter 2011-09-04 23:28:15 +00:00
• e1a3d609aa moved merger object from Segment to IndexCell to enable a correct shutdown sequence. This solves a bug where yacy cannot be shut down during an index merge that appears during the shutdown phase. orbiter 2011-09-04 23:27:12 +00:00
• 656286347e fix for javascript error during search (not ready yet) orbiter 2011-09-03 07:10:47 +00:00
• dc25c48fc9 added more libraries that are needed by solrj orbiter 2011-09-02 23:43:04 +00:00
• 2cf61a40ce fixed a bug from 7856, where Snippet returned an error by mistake when Metadata was found sixcooler 2011-09-02 16:50:05 +00:00
• 0229029dcf a bit protection against search result bugs in interactive search orbiter 2011-09-01 16:08:33 +00:00
• 610b01e1c3 - added a 'add every media object linked in a html document as a new document' to the html parser. This causes that all image, app, video or audio file that is linked in a html file is added as document. In fact that means that parsing a single html document may cause that a number of documents is inserted into the search index. - some refactoring for mime type discovery orbiter 2011-09-01 16:05:00 +00:00
• 9a8937f8b6 be more liberal when evaluating search results. This may cause that it is possible to fraud content on fresh peers, but that is better than looong waiting times for the evaluation of every link which causes that everybody rejects YaCy as 'too slow'. But this is only because of the high standards that YaCy sets to itself. If we are able to gain more users by lowering the standard, then that is useful. The option to set that flag to verify each link is still there. orbiter 2011-09-01 16:02:15 +00:00
• 3da21c4266 protection against starting of a (second) yacy peer while another one is already running on the same port orbiter 2011-09-01 13:13:21 +00:00
• 2d03dc1804 removed unnecessary warning orbiter 2011-09-01 10:37:14 +00:00
• cf8e3b0df8 small fix for count: overXX includes the count orbiter 2011-09-01 10:25:27 +00:00
• 6db8921a0f enhanced termlist orbiter 2011-09-01 10:23:22 +00:00
• b5252ef91f added new word recommendation library in DictionaryLoader_p.html orbiter 2011-09-01 10:14:17 +00:00
• 1c007188ad bugfixes in html parser orbiter 2011-08-31 16:02:06 +00:00
• b00e69c5df removed test output orbiter 2011-08-29 09:16:41 +00:00
• 231074bf0a fixed a parsing bug by reverting SVN 7766 orbiter 2011-08-28 22:59:19 +00:00
• ce11b7b6d2 *) Changed action to "" instead of "yacysearch.html". This should not do any harm, but helps a lot if the page is accessed not by its original name but by a different name which can be done by adding a symbolic link to the file system of the peer. (See http://www.yacy-forum.org/viewtopic.php?f=2&t=464) low012 2011-08-28 14:24:09 +00:00
• 30a8a2f76b *) replacing one ugly hack with an extended ugly hack ;-) low012 2011-08-28 13:32:42 +00:00
• 95379ce0b1 *) should fix some problems with RSS Importer (see http://forum.yacy-websuche.de/viewtopic.php?f=5&t=3253) low012 2011-08-28 12:59:11 +00:00
• c660f8862a *) changed links to be underlined again since lots of links were not obvious anymore *) added SVN properties low012 2011-08-28 01:14:58 +00:00
• 24e76a7b69 *) Replaced occurrences of "Wikimedia" with "MediaWiki" where applicable. (Thanks to the folks of 0x20.be for pointing this out.) *) Added description of where to place MediaWiki dump for import. low012 2011-08-28 00:16:36 +00:00
• d40a177c05 Generation Memory Strategy fine tuning add some log-output in termlist_p sixcooler 2011-08-27 15:23:24 +00:00
• 839f407fe4 Generation Memory Strategy fine tuning: - some more optimism on requests of unknown values - avoid a premature value of 0 byte available sixcooler 2011-08-25 22:32:15 +00:00
• 3e6767d66c limitation of reference evaluation (protection against crawler pits) orbiter 2011-08-25 21:12:31 +00:00
• a5541751a8 - added memory computation to termlist_p.xml - added option to delete terms in termlist_p.xml orbiter 2011-08-25 19:13:45 +00:00
• 45e497a9bd fix for term iteration orbiter 2011-08-25 18:29:30 +00:00
• 9bdee5c71c added a servlet that produces a list of term hashes that appear more than 10000 times see /api/termlist_p.xml orbiter 2011-08-25 16:49:20 +00:00
• 5dd2efc9a2 - bugfixes in html parser - new fields in solr - extended file viewer to debug parser orbiter 2011-08-25 15:52:25 +00:00
• 2c595a6a47 added new methods to count the number of objects in RWIs. lots of refactoring was necessary to introduce new Rating class and to unify naming of methods orbiter 2011-08-25 10:35:25 +00:00
• 75df87832c refactoring/better naming of methods and classes orbiter 2011-08-24 23:08:28 +00:00
• 9f9f634de2 fix in search orbiter 2011-08-24 12:12:48 +00:00
• 5f8a5ca32d - not doing merge-jobs while short on Memory - using configuration-values of crawling-max-filesize also for snippetfetching and loading files into Index sixcooler 2011-08-24 12:07:53 +00:00
• 965fabfb87 enhanced sorting speed (affects all DB operations) orbiter 2011-08-24 10:27:19 +00:00
• 41a8ee4569 added iterable implementation in KeyList orbiter 2011-08-23 20:23:40 +00:00
• 22d69a6368 refactoring in cora: added sorting package orbiter 2011-08-23 20:18:30 +00:00
• 51cf697acd refactoring: moved all score-related classes to new ranking package orbiter 2011-08-22 22:37:53 +00:00
• a0d5e7b6e6 added new score comparator orbiter 2011-08-22 22:33:08 +00:00
• 169236c6d9 almost revert changes in this class of 7880 and 7882 since MemoryControl does handle negative value requests sixcooler 2011-08-22 17:58:23 +00:00
• 4fec99115b Implementation of strategies for controlling memory resources. You can toggle between previous (standard) and new (generation) strategy at PerformanceMemory_p.html. The generation memory strategy is implemented with the objective of running more robust but with the cost of early stopping some tasks (eg. dht) while running low on memory. This new strategy does respect the generational way a heap is organized on most used jvms. These changes run fine on my 3 peers for weeks now, but as I'm human, I may fail. Please be carefull using generation memory strategy and report errors by naming OS, jvm and java_args. sixcooler 2011-08-22 17:50:03 +00:00
• 63a375b801 do not look at external dtd, cause this make this reader stay forewer(?) on on faulty dtd-locations sixcooler 2011-08-22 17:45:27 +00:00
• c39d63e7ad by default show only domain navigator orbiter 2011-08-22 10:19:15 +00:00
• 2c58af6874 - added a short memory status simulation mode - added a button in PerformanceMemory_p.html to set the simulated short memory status - bugfix: added a missing lowercase in KeyList - better concurrency in loader dispatcher orbiter 2011-08-17 22:24:17 +00:00
• c64faf41e2 addon to svn 7880 orbiter 2011-08-15 11:07:03 +00:00
• 7b7a196243 ignore cookies in httpclient per default disable cookiestore,cause the default one caused segfaults on my peers this does not harm use of cookies via YaCy as proxy sixcooler 2011-08-14 12:28:28 +00:00
• 06408a9428 since many POST-requests come as gzip they report a contentlength of -1 request memory of -1 * 3 look useless to me so I added some megs to it - even correct report of contentlength should not be harmed by this sixcooler 2011-08-13 01:04:37 +00:00
• 411ed159f8 do some extra sleep while running low on memory (1 sec. per outofmemoryCycle) sixcooler 2011-08-13 00:59:59 +00:00
• 6361f1d875 select the search window on focus so its easy to type in another query orbiter 2011-08-12 18:07:46 +00:00
• 9ab0ba41e2 using GzipDecompressingEntity from httpclient instead of our own (was just fixed there in httpclient-4.1.2 and does a proper job) sixcooler 2011-08-12 17:51:30 +00:00
• 52b477cf6f bump to httpclient-4.1.2, httpcore-4.1.3 - bugfixrelease sixcooler 2011-08-12 17:42:32 +00:00
• ca09081341 better interaction orbiter 2011-08-12 17:13:34 +00:00
• 3f0349e362 added a 'loading...' message orbiter 2011-08-12 16:09:40 +00:00
• feac494f26 switch off real-time search if index is large orbiter 2011-08-12 14:42:20 +00:00
• 07f5954570 try better handling of corrupt blobs @developer: please revert if I'm wrong see http://forum.yacy-websuche.de/viewtopic.php?f=8&t=3334 sixcooler 2011-08-12 13:27:11 +00:00
• f970670a7c - bugfix in ServerScannerList - speed up of generation of scanner list avoiding forced dns lookup orbiter 2011-08-12 13:21:18 +00:00
• 8e03b8ee8b better integration of server list in interactive search orbiter 2011-08-12 12:25:45 +00:00
• 606c5a9b40 added a serlvet that shows all scanned servers inside of the yacyinteractive search page orbiter 2011-08-12 10:31:01 +00:00
• 0a3ab7da1b do not sort concrrently the same array orbiter 2011-08-12 08:06:21 +00:00
• 77a9af99f1 same values for Xmx and Xms: memory extension may be difficult if the OS has not the remaining memory available and may kill the jvm. If the memory is reserved at the start but never used the OS may handle that as well and leave non-used space in swap area (and never swap) orbiter 2011-08-11 21:54:27 +00:00
• 594d8f546a #cccamp11 maintenance fix: anons may find up to 1000 items in interactive search (was: 100) orbiter 2011-08-11 21:37:35 +00:00
• eb14111200 encapsulate potential expensive objects in TextSnippet to allow GC them asap this reduces chance of OOMs at massive search & snippet-fetching sixcooler 2011-08-11 21:07:52 +00:00
• 3aa6528ed0 the form value was not correctly interpreted cominch 2011-08-11 07:31:35 +00:00
• 0d33cf352b removed synchronization in DNS resolve (solves a problem when loading snippets but in the past concurrent dns requests also caused deadlocks. but this is many years ago and we will give it another try) orbiter 2011-08-03 19:42:18 +00:00
• e3fc1efbef performance hack and ensuring termination in serverAccessTracker. cause: "Session_:53600#0_POST /yacy/hello.html HTTP/1.1" prio=10 tid=0x2322b000 nid=0x3ba7 runnable [0x03d3e000] java.lang.Thread.State: RUNNABLE at java.lang.Long.valueOf(Long.java:557) at de.anomic.server.serverAccessTracker.clearTooOldAccess(serverAccessTracker.java:113) at de.anomic.server.serverAccessTracker.cleanupAccessTracker(serverAccessTracker.java:75) - locked <0x3bda2ae8> (a de.anomic.server.serverAccessTracker) at de.anomic.server.serverAccessTracker.track(serverAccessTracker.java:125) at de.anomic.server.serverSwitch.track(serverSwitch.java:542) at de.anomic.http.server.HTTPDemon.parseRequestLine(HTTPDemon.java:641) at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:491) at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at de.anomic.server.serverCore$Session.listen(serverCore.java:757) at de.anomic.server.serverCore$Session.run(serverCore.java:651) orbiter 2011-08-03 18:47:43 +00:00
• 44d74f8f89 performance hacks for seed generation (because thread dumps showed multiple occurrences at these code points) orbiter 2011-08-03 18:32:11 +00:00
• 5cd07d7f84 early freeing resources on deleting index reference if search-verification fails (aka Switchboard.cleanupJob) sixcooler 2011-08-02 15:52:33 +00:00
• a311596881 finishing up my commits (7855-7858) which could be helpful for sixcooler 2011-08-01 23:35:24 +00:00
• 9170a434ed throwing an exception again in FileUtils.copy(reader, writer) OOMs could occour here and should not be ignored sixcooler 2011-08-01 23:32:58 +00:00
• c0caca57e3 stoping thread for fetching searchresults if running short on memory - in most cases at least one thread stays alive for getting the results - fewer threads should do the work with less resouces, but much slower then sixcooler 2011-08-01 23:32:29 +00:00
• ce248cc8dd less byte-arrays of response-content, less byte-array <-> stream conversation sixcooler 2011-08-01 23:31:08 +00:00
• 59b767eebd stop loading via http at defined maximum of bytes - even size is unknown before loading sixcooler 2011-08-01 23:28:23 +00:00
• e131bf804d correct value @ low memory warning see: http://forum.yacy-websuche.de/viewtopic.php?f=5&t=3317 sixcooler 2011-07-26 01:05:28 +00:00
• 48560a44a9 bump to httpcore-4.1.2: a bugfixrelease sixcooler 2011-07-25 00:48:29 +00:00
• 916d79111e Runtime.maxMemory() DOES change @ runtime: I wondered getting Total-ram > Max-ram and MemoryControl.available() < 0 MemoryControl.available() < 0 causes some errors where its value is used for dimension of buffers for eg. sixcooler 2011-07-19 12:48:50 +00:00
• 3a5fa73008 * revert parts of previous commit, because it breaks the trickle-feature f1ori 2011-07-19 12:04:40 +00:00
• 6e79675ff3 * use gzip-encoding in more cases * send Expire-Header for static content * should improve webserver-performance for slow connections * fixes #37 f1ori 2011-07-19 11:47:53 +00:00
• 299af4943c added another memory protection hack orbiter 2011-07-17 17:55:08 +00:00