yacy_search_server

Mirrors/yacy_search_server

Fork 0

mirror of https://github.com/yacy/yacy_search_server.git synced 2025-07-22 09:14:38 -04:00

Commit Graph

Select branches

Hide Pull Requests

master

#1

#10

#100

#100

#101

#101

#104

#105

#108

#110

#114

#12

#122

#127

#127

#128

#135

#137

#137

#14

#144

#149

#151

#154

#155

#157

#159

#167

#167

#18

#183

#186

#188

#2

#21

#218

#219

#220

#226

#23

#238

#239

#240

#250

#258

#26

#260

#261

#27

#278

#28

#283

#29

#292

#3

#303

#32

#325

#327

#332

#333

#334

#338

#34

#355

#356

#357

#37

#38

#389

#39

#390

#4

#40

#402

#403

#404

#405

#408

#41

#41

#419

#42

#420

#436

#437

#439

#44

#440

#442

#443

#447

#448

#45

#45

#46

#461

#462

#463

#464

#47

#472

#478

#484

#485

#488

#489

#490

#495

#5

#5

#51

#52

#53

#533

#535

#54

#540

#541

#55

#553

#554

#556

#559

#56

#560

#562

#564

#567

#568

#575

#576

#58

#580

#583

#584

#585

#588

#589

#594

#596

#597

#598

#60

#600

#603

#605

#607

#61

#610

#612

#614

#615

#616

#617

#619

#62

#621

#628

#633

#642

#649

#65

#650

#651

#657

#66

#661

#661

#664

#666

#67

#670

#672

#678

#68

#682

#686

#688

#689

#69

#690

#692

#693

#694

#695

#696

#697

#698

#699

#700

#704

#71

#710

#710

#715

#718

#72

#76

#77

#78

#8

#88

#9

#9

#91

#93

#93

#97

#97

#98

0.99

1.0

Release_1.0

Release_1.01

Release_1.02

Release_1.03

Release_1.04

Release_1.1

Release_1.2

Release_1.3

Release_1.4

Release_1.5

Release_1.6

Release_1.68

Release_1.7

Release_1.72

Release_1.80

Release_1.82

Release_1.90

Release_1.92

7b1f5b0430 - better media search ranking - better concurrency with enhanced synchronization in sort stack orbiter 2009-11-20 13:19:12 +00:00
4df88a4e7a - fixes for missing or bad hashCode computation - fixes for bad equals() methods that had not been used by hash maps and therefore some classes did not work as objects in hash maps. - this may also affect some cases where double-checks should have been, but did not work. orbiter 2009-11-20 12:11:56 +00:00
dbdf2570ba added comparator and more fixes for SortStack/SortStore orbiter 2009-11-20 03:30:48 +00:00
d2938c44a1 - added bmp parser to the document parsers - image parser that implement the document parser interface return itself in the list of images of the document which should cause that the parsed images contribute to the image search orbiter 2009-11-19 23:22:53 +00:00
1dff620181 Better implementation of SortStack and SortStore and adoptions in all using classes to implement the necessary Comparable interface and hash code computation. The better SortStack performance affects crawling and image search speed and quality. orbiter 2009-11-19 13:49:28 +00:00
fe41a84330 some enhancements in web caching: avoid double loading of response metadata and/or content orbiter 2009-11-19 10:17:26 +00:00
06d0dcde20 more enhancements to image search orbiter 2009-11-19 00:43:42 +00:00
4c6312d103 enhanced image search orbiter 2009-11-18 23:56:05 +00:00
2d8f3ee301 some performance hacks orbiter 2009-11-18 16:03:28 +00:00
36fbfdcb21 more performance for remote search orbiter 2009-11-18 15:13:06 +00:00
5c7b32a4fa better performance for list api (blacklist transfer) orbiter 2009-11-18 15:11:52 +00:00
94b2a664f3 - use a static DiskFileItemFactory (one instantiation is enough) - use more memory for the DiskFileItemFactory to avoid IO when POST commands come orbiter 2009-11-18 15:05:51 +00:00
267108470f testing jmx console for yacy: - start YaCy with startYACY.sh -l - open jconsole and open localhost:9999 orbiter 2009-11-18 14:54:30 +00:00
d9835d8568 Conversion of the de. Language to UTF-8 format. This version will replace the now used language File in the future. Please test and report bugs to me (th-suess@gmx.de). suessthomas 2009-11-17 16:19:46 +00:00
fd0658ce7c avoid forced execution of InetAddress.getLocalHost() at startup, because that hangs at some strangely declared linux configurations. The Domains.localHostAddresses object is first instantiated with a more simple logic and enriched with more host addresses using a concurrent thread that will not block a startup process. orbiter 2009-11-16 23:08:20 +00:00
013f337d3f - avoid unnecessary host name lookups for localhost - avoid unnecessary reverse domain name lookups for remote access orbiter 2009-11-16 23:00:54 +00:00
141712ec95 *) small changes to UI *) password will not be deleted anymore when changing to unlimited access from localhost low012 2009-11-15 22:15:25 +00:00
12dd8ece3e enabled memory protection from 6459 with 50000kb (disables dht-in) this should only apply if there is really little memory available because it is checked by threads explictly requesting memory lotus 2009-11-13 16:26:45 +00:00
20c5d78a5c fix for a ConcurrentModificationException orbiter 2009-11-11 23:31:12 +00:00
5afd9f7a91 fix for crlf writing orbiter 2009-11-11 22:50:44 +00:00
7144d2df6e added crawlReceipt servlet as individual class to examine OOM problem as documented in http://forum.yacy-websuche.de/viewtopic.php?p=18120#p18120 orbiter 2009-11-11 16:12:00 +00:00
2d3c98b742 less computation within synchronized blocks orbiter 2009-11-11 16:07:40 +00:00
1a146b0d73 added a patch to ignore bad mime-ignore patterns orbiter 2009-11-11 15:49:53 +00:00
29fe436e36 - fixed post-ranking including prefer mask - enhanced a core database access method / less wasted ram orbiter 2009-11-09 19:14:51 +00:00
e9ab130ad7 fixed start/stop using ant orbiter 2009-11-09 19:12:33 +00:00
5399d1e2bc refactoring (reason: get more abstraction to use the blacklist class; for integration in other servlets) orbiter 2009-11-08 22:58:57 +00:00
a97fdb4566 catch for NPE in image parser orbiter 2009-11-07 23:39:31 +00:00
9ee7862710 *) added configuration script low012 2009-11-07 13:21:27 +00:00
534182559c removed concurrency hacks from SplitTable because it showed deadlock-like situation. see thread dump at http://forum.yacy-websuche.de/viewtopic.php?p=18081#p18081 orbiter 2009-11-07 11:52:03 +00:00
1fa0ac26e9 better protection against NPEs during search/ranking orbiter 2009-11-07 10:58:33 +00:00
2bab0679e0 lost my key :-( orbiter 2009-11-06 23:46:29 +00:00
4c99d4683d possible fix for lost crawl profile handles: clean-up job did wrong measurement to see if crawl is still running. orbiter 2009-11-06 23:15:20 +00:00
cd6745b292 accept rss feeds without channel descriptions orbiter 2009-11-06 22:46:21 +00:00
08f1cbb125 another update to the pdf parser orbiter 2009-11-06 22:41:37 +00:00
54c54fb144 get a handle for grep: 'StackTrace' orbiter 2009-11-06 19:55:21 +00:00
605e896d6c more details for exception catching when parsing pdfs orbiter 2009-11-06 19:47:24 +00:00
18b21eaffe small fixes to search default values and server logging orbiter 2009-11-06 19:13:35 +00:00
6edc168cfe option to disable dht by memory limit: memory.acceptDHT in kbytes not yet pre-enabled, will clear on every startup please review since this could break dht in freeworld lotus 2009-11-06 19:13:30 +00:00
4431b9767e added about 450 replacements for printStackTrace() methods to pipe such traces into the log at DATA/LOG/ orbiter 2009-11-05 20:28:37 +00:00
e3025ee691 - new icon for OAI-PMH loading action - added many stack trace outputs for exceptions in crawl profile handler to find the 'missing profile handle' bug - catched one more timeout exception in httpd file loader orbiter 2009-11-05 16:40:15 +00:00
f0b8db93f0 - more abstraction of serverCore thread access - no more keep-alive when number of connections exceeds 1/2 of the allowed number of connection orbiter 2009-11-05 14:54:43 +00:00
19f31bb043 - moved OAI-PMH source list file from SETTINGS to DICTIONARIES/harvesting - added convenience method for loading of files from the web in LoaderDispatcher orbiter 2009-11-04 16:15:28 +00:00
2889b9426e missing code for last commit orbiter 2009-11-04 12:03:19 +00:00
b6a8887ff5 better handling of running sessions without explicit hashtable orbiter 2009-11-04 11:59:15 +00:00
1dc7ea986a added a dynamic keep-alive time-out for http server sessions: if there are many concurrent server sessions, the timout is decreased. This should avoid a situation where the clean-up thread is too late to stop running http sessions that should be terminated before the maximum number of server sessions is reached. orbiter 2009-11-04 11:01:09 +00:00
e77c906673 *) minor changes mainly in comments *) added svn:keyword settings for several files low012 2009-11-03 22:47:53 +00:00
f1740edbf8 *) added skript to change memory settings, password and port (experimental, don't blame me if it messes up your configuration) *) minor change in Digest class, added option in main method, might not be optimal yet low012 2009-11-03 22:28:29 +00:00
11f7da06ed - fixes to csv parser - automatic OAI-PMH import by just clicking on one link from the provided resource list orbiter 2009-11-03 21:18:19 +00:00
9b6762ec2e - added a csv "comma separated values" parser to parse OAI-PMH sources from http://roar.eprints.org/index.php?action=csv - integrated the csv parser into the crawlers parser list - added an extension to the OAI-PMH import function to download and show the roar csv file using the csv parser orbiter 2009-11-03 20:10:59 +00:00
0f63de8236 - it is now possible to start several OAI imports concurrently (still not possible to start them with one single request, that will be next) - added a monitor for all running and finished OAI imports (with a little bit of animation..) orbiter 2009-11-03 16:15:22 +00:00
176e334aa4 fixes orbiter 2009-11-02 19:23:05 +00:00
2fa6bf440b workflow update to OAI-PMH importer orbiter 2009-11-02 18:19:30 +00:00
b0b7a4f9a5 - added function to OAI-PMH reader that can pull all records from a server using an evaluation of the resumption token to get URL to retrieve remaining records - added monitoring for retrieved records orbiter 2009-11-02 11:53:14 +00:00
350d13e153 very first working version of oai-pmh importer: if given the right url, the importer can read and index listRecord xml files and calculate the right resumptionURL which is then given as next default start point for the importer url input. no automatic harvesting by now, this will be done later orbiter 2009-11-02 00:14:14 +00:00
58616d99e4 patch for yacy disk usage detection on lvm host by Michael S. lotus 2009-11-01 08:54:16 +00:00
79251e6f60 configurable disk space hardlimit for dht lotus 2009-10-31 19:12:53 +00:00
a0e891c63d - some redesign in UI menu structure to make room for new 'Content Integration' main menu containing import servlets for Wikimedia Dumps, phpbb3 forum imports and OAI-PMH imports - extended the OAI-PMH test applet and integrated it into the menu. Does still not import OAI-PMH records, but shows that it is able to read and parse this data - some redesign in ZURL storage: refactoring of access methods, better concurrency, less synchronization - added a limitation to the LURL metadata database table cache to 20 million entries: this cache was until now not limited and only limited by the available RAM which may have caused a memory-leak-like behavior. orbiter 2009-10-31 11:58:06 +00:00
8a1046feaa less maximum file size, too many problems with larger size orbiter 2009-10-30 20:21:45 +00:00
4240785f20 added anti-alias function for line drawing orbiter 2009-10-30 15:58:36 +00:00
30f108f97d added stub of oai-pmh importer (not working yet) orbiter 2009-10-30 15:58:04 +00:00
77c99e500f added more control over memory allocation should avoid some of the OOMs orbiter 2009-10-27 15:25:48 +00:00
52470d0de4 - fix for xls parser - fix for image parser - temporary integration of images as document types in the crawler and indexer for testing of the image parser orbiter 2009-10-22 22:38:04 +00:00
5e8038ac4d - refactoring of blacklists - refactoring of event origin encoding orbiter 2009-10-21 20:14:30 +00:00
26fafd85a5 - more refactoring - fixed problem with parsers orbiter 2009-10-21 15:12:34 +00:00
e48f3dfb1e added documentation for new yacy package structure orbiter 2009-10-20 12:05:36 +00:00
3528b970d6 - refactoring - added new experimental (not-yet-working) image parser - added new test image orbiter 2009-10-19 22:34:44 +00:00
6414ac9ecf fix for debian int script http://forum.yacy-websuche.de/viewtopic.php?t=2418 lotus 2009-10-19 17:05:15 +00:00
63e489c5f7 removed win9x scripts because the latest jre has v1.3 for these systems http://www.java.com/en/download/help/win95.xml lotus 2009-10-18 09:45:34 +00:00
cde1611919 updated junit orbiter 2009-10-18 02:52:09 +00:00
a8ce192f63 - shifted main classes to new package net.yacy - fixed some bugs in last commit orbiter 2009-10-18 01:38:07 +00:00
b79f4f062f refactoring of yacy documents and parsers: they depend now only on the kelondro classes orbiter 2009-10-18 00:53:43 +00:00
0fd9540866 Configuration of HTTPDProxyHandler logging hermens 2009-10-17 14:04:18 +00:00
519c3619ff *) minor changes low012 2009-10-17 00:32:07 +00:00
f5656b2ae1 *) Made sure that only files with appropriate file endings are listed as skin or language files. *) Introduced protection against directory traversal attacks in configuration servlets for skin and language configuration. Files can only be deleted if they are contained in a list of files which has been read by the servlet first. low012 2009-10-17 00:26:14 +00:00
3434ca381f *) grrr low012 2009-10-16 22:17:21 +00:00
ae42c51cf7 *) Skin names and language names are displayed in alphabetical order in dropdown menu now. low012 2009-10-16 22:16:36 +00:00
56a5bd090d Small fixes to header.template for more XHTML compatibility. suessthomas 2009-10-16 20:31:06 +00:00
34c71b22e8 fix and enable parser unit tests (tested with eclipse) f1ori 2009-10-16 09:33:18 +00:00
99683f5f11 small changes to green color and round corners orbiter 2009-10-15 19:39:11 +00:00
76bca8cffd show interactive search without menu orbiter 2009-10-15 13:26:14 +00:00
3d5eeb842a new default skin 'pdblue' The old default skin named 'default' is renamed to 'classic-blue'. All users will keep their current default skin named default, but YaCy will copy the classic-blue also to the skin folder. For all new peers, the new skin pdblue is used. orbiter 2009-10-15 12:59:44 +00:00
cee7a05ff2 - de-serialized the pdf parser - added fail callback for file indexer orbiter 2009-10-15 10:47:29 +00:00
9db928ce53 replaced fontbox 0.7.3 with fontbox 0.8.0 orbiter 2009-10-15 09:51:16 +00:00
c2272785c7 - fix for xlsx and pptx parsing - less exception logging for swf parser orbiter 2009-10-14 19:15:38 +00:00
afae2a0bee Small changes to the Yacy Skins. suessthomas 2009-10-14 19:11:52 +00:00
0975b1b493 update for apache poi library possible solves http://forum.yacy-websuche.de/viewtopic.php?p=17736#p17736 lotus 2009-10-14 15:24:53 +00:00
c864901087 - moved httpd.mime to defaults path - some documentation fixes - adopted a default setting for the search window: moves css setting to base.css - some enhancements for the DocumentIndex class orbiter 2009-10-14 13:29:09 +00:00
8829ec5f18 *) made sure that   is replaced with a space and not just deleted in CharacterCoding.java *) added annotations and made minor changes to serverObjects.java *) set subversion properties for several files low012 2009-10-13 20:57:56 +00:00
6c347a37eb more options for DocumentIndex orbiter 2009-10-13 08:43:02 +00:00
6192205533 more final modifier orbiter 2009-10-12 21:59:39 +00:00
0f6b011e1a fix for new index location and better way to use own classes by reflection orbiter 2009-10-12 21:12:42 +00:00
7a3bbd950f :-( orbiter 2009-10-12 20:29:03 +00:00
b953f04f90 one more reflection fix orbiter 2009-10-12 17:45:42 +00:00
77d6604856 fix for npe, see http://forum.yacy-websuche.de/viewtopic.php?p=17727#p17727 orbiter 2009-10-12 17:41:16 +00:00
2a7fe35f92 performance tuning using more final modifiers in the kelondro core orbiter 2009-10-12 17:37:12 +00:00
cb4de9ceee fixed a bug in table iterator (did not recognize elements in write buffer) orbiter 2009-10-12 08:06:35 +00:00
5841ee83d3 refactoring orbiter 2009-10-11 21:29:18 +00:00
e7f18ba24b refactoring orbiter 2009-10-11 00:24:42 +00:00
ce8dc575ca refactoring orbiter 2009-10-11 00:12:19 +00:00
bea3b99aff moved table and util classes orbiter 2009-10-10 01:14:19 +00:00