Commit Graph

Select branches

Hide Pull Requests

master

#1

#10

#100

#100

#101

#101

#104

#105

#108

#110

#114

#12

#122

#127

#127

#128

#135

#137

#137

#14

#144

#149

#151

#154

#155

#157

#159

#167

#167

#18

#183

#186

#188

#2

#21

#218

#219

#220

#226

#23

#238

#239

#240

#250

#258

#26

#260

#261

#27

#278

#28

#283

#29

#292

#3

#303

#32

#325

#327

#332

#333

#334

#338

#34

#355

#356

#357

#37

#38

#389

#39

#390

#4

#40

#402

#403

#404

#405

#408

#41

#41

#419

#42

#420

#436

#437

#439

#44

#440

#442

#443

#447

#448

#45

#45

#46

#461

#462

#463

#464

#47

#472

#478

#484

#485

#488

#489

#490

#495

#5

#5

#51

#52

#53

#533

#535

#54

#540

#541

#55

#553

#554

#556

#559

#56

#560

#562

#564

#567

#568

#575

#576

#58

#580

#583

#584

#585

#588

#589

#594

#596

#597

#598

#60

#600

#603

#605

#607

#61

#610

#612

#614

#615

#616

#617

#619

#62

#621

#628

#633

#642

#649

#65

#650

#651

#657

#66

#661

#661

#664

#666

#67

#670

#672

#678

#68

#682

#686

#688

#689

#69

#690

#692

#693

#694

#695

#696

#697

#698

#699

#700

#704

#71

#710

#710

#715

#718

#72

#721

#722

#730

#734

#737

#76

#77

#78

#8

#88

#9

#9

#91

#93

#93

#97

#97

#98

0.99

1.0

Release_1.0

Release_1.01

Release_1.02

Release_1.03

Release_1.04

Release_1.1

Release_1.2

Release_1.3

Release_1.4

Release_1.5

Release_1.6

Release_1.68

Release_1.7

Release_1.72

Release_1.80

Release_1.82

Release_1.90

Release_1.92

Mono Color

• 8e56c2ace6 fix for fixes from this afternoon orbiter 2009-08-07 22:53:49 +00:00
• cf739edc2e fix for possible deadlock, see http://forum.yacy-websuche.de/viewtopic.php?p=17017#p17017 orbiter 2009-08-07 12:11:22 +00:00
• 6354b5e447 removed possible deadlock, see http://forum.yacy-websuche.de/viewtopic.php?p=17017#p17017 orbiter 2009-08-07 12:04:14 +00:00
• 5cc17ccf8a a better caching with less overhead and more appropriate synchronisation use in more than 10 different data objects orbiter 2009-08-07 11:55:32 +00:00
• 2e01bd955d wrong display of hints / hints wrong / incomplete orbiter 2009-08-07 11:32:41 +00:00
• 39ae96450b draw more peers in network picture orbiter 2009-08-07 08:36:15 +00:00
• 92edd24e70 fixed problem with switching of networks orbiter 2009-07-30 15:49:23 +00:00
• 0575f12838 fix for deadlock orbiter 2009-07-30 09:08:44 +00:00
• fbfdaf063d - patch to omit IndexOutOfBoundsException when a b64-encoded key appears not to be well-formed. In that case the key is still accepted but rated higher than other regular keys to create a virtual ordering between well-formed and ill-formed keys - check routine at the beginning of the import of table keys that check that all imported keys are well-formed. All records that have a ill-formed key are deleted. This is a hack and is not tested since I don't have bad data here to test with. If the effect is seen in the wild, please report in the forum. orbiter 2009-07-29 19:43:11 +00:00
• 65b1d51e70 added xml version of windows office test files orbiter 2009-07-27 12:45:15 +00:00
• c0e17de2fb - fixes for some problems with the new crawling/caching strategies - speed enhancements for the cache-only cache policy by using special no-delay rules in the balancer - fixed some deadlock- and 100% CPU problems in the balancer orbiter 2009-07-25 21:38:57 +00:00
• 634a01a9a4 replaced wget-requests with caching requests orbiter 2009-07-24 14:52:27 +00:00
• c6c97f23ad - added cache usage properties to crawl start - added special rule to balancer to omit forced delays if cache is used exclusively - extended the htCache size by default to 32GB orbiter 2009-07-24 11:54:04 +00:00
• c4ae2cd03f fixed bug that caused deletion of crawl profiles at every application startup orbiter 2009-07-23 22:09:02 +00:00
• 161d2fd2ef redesign of access to the HTCache (now http.client.Cache): - better control to the cache by using combined request-header and content access methods - refactoring of many classes to comply to this new access method - make shure that the cache is always written if something was loaded - some redesign of the process how http response results are feeded into the new indexing queue - introduction of a cache read policy: * never use the cache * use the cache if entry exist * use the cache if the proxy freshness rule confirmes * use only the cache and go never online - added configuration options for the crawl profiles to use the new cache policies. There is not yet a input during crawl start to set the policy but this will be added in another step. - set the default policies for the existing crawl profiles. If you want them to appear in your default profiles you must delete the crawl profiles database; othervise the policy is 'proxy freshness rule' - enhanced some cache access methods in such a way that unnecessary retrievals are omitted (i.e. for size computation). That should reduce some IO but also a lot of CPU computation because sizes were computed after decompression of content after retrieval of the content from the disc. orbiter 2009-07-23 21:31:51 +00:00
• da43164dd6 fix for UNRESOLVED_PATTERN see http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2300 lulabad 2009-07-23 06:02:36 +00:00
• d7c9c765bb changes by Thomas Süß daburna 2009-07-22 17:24:04 +00:00
• ba2e6de538 fix empty version string again f1ori 2009-07-21 19:56:40 +00:00
• 53081ee6da changes by Thomas Süß daburna 2009-07-21 12:22:05 +00:00
• 51534df0cb fix for possible synchronization problem see also: http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2292&hilit=&p=16787#p16787 orbiter 2009-07-20 08:21:17 +00:00
• 4da9042e8a code simplification orbiter 2009-07-19 21:59:29 +00:00
• 1d8d51075c refactoring: - removed the plasma package. The name of that package came from a very early pre-version of YaCy, even before YaCy was named AnomicHTTPProxy. The Proxy project introduced search for cache contents using class files that had been developed during the plasma project. Information from 2002 about plasma can be found here: http://web.archive.org/web/20020802110827/http://anomic.de/AnomicPlasma/index.html We stil have one class that comes mostly unchanged from the plasma project, the Condenser class. But this is now part of the document package and all other classes in the plasma package can be assigned to other packages. - cleaned up the http package: better structure of that class and clean isolation of server and client classes. The old HTCache becomes part of the client sub-package of http. - because the plasmaSwitchboard is now part of the search package all servlets had to be touched to declare a different package source. orbiter 2009-07-19 20:37:44 +00:00
• 67da20647f * add new odf parser based on sax-xml-parser * remove odf_utils-jar * test metadata in ParserTest f1ori 2009-07-18 15:04:34 +00:00
• de4f0a006f removed superfluous windows target lotus 2009-07-18 14:46:42 +00:00
• 06557485f5 * added parser unittest! f1ori 2009-07-17 22:03:34 +00:00
• 69dfd03985 reactivate unittests * fix old tests * add buildtarget "ant test" f1ori 2009-07-17 20:58:21 +00:00
• 6d0e6d591b * ops, fix compiler error :( f1ori 2009-07-17 20:02:56 +00:00
• 3e5beb1654 * fix for empty version in seedlist f1ori 2009-07-17 19:16:26 +00:00
• 5bb8074150 removed the indexing queue. This queue was superfluous since the introduction of the blocking queues last year, where documents are parsed, analysed and stored in the index with concurrency. - The indexing queue was a historic data structure that was introduced at the very beginning at the project as a part of the switchboard organisation object structure. Without the indexing queue the switchboard queue becomes also superfluous. It has been removed as well. - Removing the switchboard queue requires that all servlets are called without a opaque generic ('<?>'). That caused that all serlets had to be modified. - Many servlets displayed the indexing queue or the size of that queue. In the past months the indexer was so fast that mostly the indexing queue appeared empty, so there was no use of it any more. Because the queue has been removed, the display in the servlets had also to be removed. - The surrogate work task had been a part of the indexing queue control structure. Without the indexing queue the surrogates needed its own task management. That has been integrated here. - Because the indexing queue had a special queue entry object and properties attached to this object, the propterties had to be moved to the queue entry object which is part of the new indexing queue withing the blocking queue, the Response Object. That object has now also the new properties of the removed indexing queue entry object. orbiter 2009-07-17 13:59:21 +00:00
• 597393db3b changed default visibility of classes/objects in upnp lib (eclipse tells me that this would improve performance, however, this removes compiler warnings) orbiter 2009-07-16 12:19:40 +00:00
• eea4c17ef2 removed rpm parser - no-one used that thing - loading huge rpm files bay be causes for crashes orbiter 2009-07-16 11:06:49 +00:00
• b332dfad67 - inserted request object into response object which carries this now instead generating new objects - fixed a problem with the crawler introduced in SVN 6216 orbiter 2009-07-15 23:08:35 +00:00
• ca72ed7526 -removed superfluous crawl cache -refactoring of crawler classes orbiter 2009-07-15 21:07:46 +00:00
• 8103ccec4c removed compiler warnings in imported classes orbiter 2009-07-15 20:44:23 +00:00
• 52e371b8f7 suppress warnings for upnplib code lotus 2009-07-15 16:22:56 +00:00
• 477807e0e6 * updated jxpath to latest v1.3 * added upnplib as source without packages: jmx remote samples lotus 2009-07-15 16:13:24 +00:00
• 049fb23a8d removed unused/unsupported ant targets orbiter 2009-07-15 14:16:25 +00:00
• 13c63f4082 a set of small fixes to crawling behaviour orbiter 2009-07-15 14:15:51 +00:00
• a564df3984 update to mime types in parsers and httpd.mime orbiter 2009-07-15 14:10:29 +00:00
• 43c8defd79 enhanced parser with more extension + mime attributes orbiter 2009-07-14 13:32:53 +00:00
• aee35bff6f replaced StringBuffer with StringBuilder in tar lib orbiter 2009-07-14 13:31:57 +00:00
• 49bbb9bd45 replaced tar library with integrated apache ant tar lib orbiter 2009-07-14 11:31:40 +00:00
• f987fc6b4a added tar classes from apache ant tools orbiter 2009-07-14 11:25:40 +00:00
• f2d4b6d7fa added tar classes from apache ant tools orbiter 2009-07-14 11:25:05 +00:00
• b2263bc720 enhanced document type recognition orbiter 2009-07-14 11:01:05 +00:00
• aa38eb5a20 * maxfilesize -1 for infinite filesize lotus 2009-07-14 08:39:39 +00:00
• 7d493cf8cc moved parser configuration in separate servelet orbiter 2009-07-14 06:57:13 +00:00
• 9cfe89c8fc * process content-length as soon as it is received * corrected indentation lotus 2009-07-13 19:55:13 +00:00
• 5240d22773 removed unused library jsmooth orbiter 2009-07-13 18:16:03 +00:00
• 3d26161dd1 removed unused libraries orbiter 2009-07-13 14:47:09 +00:00
• 50cf80056f removed jmimemagic library orbiter 2009-07-13 10:58:37 +00:00
• e3c7f61145 removed unused libraries orbiter 2009-07-13 10:21:22 +00:00
• 3f113f38a8 removed unused imports removed unused libs from eclipse class path orbiter 2009-07-13 10:19:10 +00:00
• 9f083bb6b2 check filetype before loading (no more mp4 loading) lotus 2009-07-12 16:50:11 +00:00
• b118bdd994 *) Deleted obsolete license file. low012 2009-07-12 16:38:13 +00:00
• 076ae02c44 * added pl and py to extensions excepted by htmlParser f1ori 2009-07-12 16:35:35 +00:00
• d5e51cfd09 * workaround for non-working build property replacements f1ori 2009-07-12 09:38:03 +00:00
• f814e0fa81 enable warnings and fix most of it f1ori 2009-07-11 21:01:27 +00:00
• 8931c8d6b4 improvments to debianpackage: * autoupdate completely disabled, display hint * restart-button in interface works! f1ori 2009-07-11 17:03:22 +00:00
• fc1dc38b55 *) added spaces to make sure that no words are concatinated by accident low012 2009-07-10 22:51:53 +00:00
• f242e7d7bc *) using Apache POI library to parse Word documents now *) removed tm-extractors library (can be found at http://www.textmining.org/ if necessary again) low012 2009-07-10 22:49:16 +00:00
• caedd72400 - enhanced logging and exception details for parsers - removed inconsistencies in mime type declaration (one mime type should only appear once in all parsers) orbiter 2009-07-10 16:15:33 +00:00
• 4b74ad0a46 fixed setting of parser configuration servlets orbiter 2009-07-10 15:02:34 +00:00
• 57a88d435b redesign of parser mime type detection and parser steering There is now a mime-blacklist instead of a mime-whitelist orbiter 2009-07-10 14:22:17 +00:00
• e15d27bc63 avoiding double/wrong parser errors lotus 2009-07-10 09:31:10 +00:00
• 21b8704fb4 refactoring of the ParserDispatcher and ParserConfig: resulted into Idiom, Parser and Classification classes orbiter 2009-07-09 22:25:31 +00:00
• 8ca1f5d400 - some work to integrate the html parser the same way as the other parsers are integrated (not finished) - added migration of code of settings pages (hmm.. does not work correctly yet, sorry) - more refactoring - removed more unused code orbiter 2009-07-09 20:56:30 +00:00
• 1ee109761f *) added changes which were lost *) additional annotations *) additional svn properties *) _no_ functional changes low012 2009-07-09 20:13:11 +00:00
• 499723891d removed all non-http daemons; they had not been used and may be a potential security risk. orbiter 2009-07-08 22:24:34 +00:00
• 0e8647d62f refactoring of search classes orbiter 2009-07-08 22:14:57 +00:00
• 92be8f39bf removed unused packages orbiter 2009-07-08 22:08:35 +00:00
• dafffd0153 refactoring of parsers and document processing orbiter 2009-07-08 21:48:08 +00:00
• 8041e91f56 *) Ooops! low012 2009-07-08 20:58:47 +00:00
• 69551ff3d9 *) added several MIME types (derived from http://filext.com/), some of them might be rather uncommon *) added an annotation forgotten in last commit low012 2009-07-08 20:53:08 +00:00
• 11dfb2d54f minor changes: *) added annotations *) set svn properties and added keywords to comments of parser classes *) made a variable final to prevent (theoretical case of) change of object instance in synchronized block low012 2009-07-08 19:20:03 +00:00
• ba7905aaa8 PortalSearch: - removed buttons - auto load on scroll - some small fixes apfelmaennchen 2009-07-08 19:01:06 +00:00
• 77d2a3782c removed strange debugging strings orbiter 2009-07-06 15:21:43 +00:00
• 4320f69574 universal handling for crashed parsers reverting r6090/1 lotus 2009-07-05 08:12:14 +00:00
• 535aee4425 update for windows build lotus 2009-07-05 08:09:50 +00:00
• 801aa08162 added f1oris update location orbiter 2009-07-04 20:56:25 +00:00
• 024744245c small refactoring to prepare for new queues orbiter 2009-07-04 12:17:10 +00:00
• 16efcd0366 fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2252&hilit=&p=16389#p16389 orbiter 2009-07-04 06:24:58 +00:00
• 0f3246e90a * fix debian package * add Class containing buildvariables f1ori 2009-07-03 21:57:05 +00:00
• 8544cfd5a6 * remove seperate build-files for parsers f1ori 2009-07-03 20:06:06 +00:00
• 24cb6d68bc - renamed Stack to RecordStack to avoid name confusion with new classes - added new Stack class that implements a stack on BLOB files - added new Stacks class that can be used for a set of Stacks (a 'Stack Database') - added methods to other classes to support the new stacks orbiter 2009-07-03 16:35:34 +00:00
• ec2970cc76 higher dht distribution speed by default lotus 2009-07-03 13:28:19 +00:00
• 995da28c73 all stack/heap files that had been stored in DATA/PLASMA are now stored in the network-specific QUEUES path There is no migration. All crawls must be restarted. orbiter 2009-07-02 17:01:23 +00:00
• aac89bf8ca trying to avoid "exceeding limit" message of server orbiter 2009-07-02 15:47:42 +00:00
• d553e4ff39 added visio test files and mime types orbiter 2009-07-02 15:17:39 +00:00
• 48d78166ed * fix double copy of libraries f1ori 2009-07-02 12:33:47 +00:00
• 65888a001e workaround for missing libs in ant dist now there are some files double :-( please help! orbiter 2009-07-02 11:18:01 +00:00
• 7f868ca3c2 resource observer: support for yacyroot\DATA on an NTFS hardlink (Windows) lotus 2009-07-02 10:02:20 +00:00
• 409538e17a code cleanup and code simplifcation orbiter 2009-06-30 22:20:55 +00:00
• 160031758d fix for problem with initializer orbiter 2009-06-30 21:57:32 +00:00
• 6fc18b9d51 removed unused ant targets orbiter 2009-06-30 15:01:42 +00:00
• 302a02cec8 moved all libraries from libx to lib removed libx directory all libraries are now in lib, instead the test libraries in libt which are not part of releases orbiter 2009-06-30 14:55:23 +00:00
• 1f1399e5c5 extending visibility of objects and methods to avoid synthetic accessor methods and increase performance orbiter 2009-06-30 13:25:46 +00:00
• 154bbc3364 code cleanup: call of static methods directly to the class orbiter 2009-06-30 13:01:35 +00:00
• 222850414e simplification of the code: removed unused classes, methods and variables orbiter 2009-06-30 09:27:46 +00:00
• 93dfb51fd4 problems with code style orbiter 2009-06-29 22:22:35 +00:00