yacy_search_server

Mirrors/yacy_search_server

Fork 0

mirror of https://github.com/yacy/yacy_search_server.git synced 2025-11-06 23:56:13 -05:00

Commit Graph

Select branches

Hide Pull Requests

master

#1

#10

#100

#100

#101

#101

#104

#105

#108

#110

#114

#12

#122

#127

#127

#128

#135

#137

#137

#14

#144

#149

#151

#154

#155

#157

#159

#167

#167

#18

#183

#186

#188

#2

#21

#218

#219

#220

#226

#23

#238

#239

#240

#250

#258

#26

#260

#261

#27

#278

#28

#283

#29

#292

#3

#303

#32

#325

#327

#332

#333

#334

#338

#34

#355

#356

#357

#37

#38

#389

#39

#390

#4

#40

#402

#403

#404

#405

#408

#41

#41

#419

#42

#420

#436

#437

#439

#44

#440

#442

#443

#447

#448

#45

#45

#46

#461

#462

#463

#464

#47

#472

#478

#484

#485

#488

#489

#490

#495

#5

#5

#51

#52

#53

#533

#535

#54

#540

#541

#55

#553

#554

#556

#559

#56

#560

#562

#564

#567

#568

#575

#576

#58

#580

#583

#584

#585

#588

#589

#594

#596

#597

#598

#60

#600

#603

#605

#607

#61

#610

#612

#614

#615

#616

#617

#619

#62

#621

#628

#633

#642

#649

#65

#650

#651

#657

#66

#661

#661

#664

#666

#67

#670

#672

#678

#68

#682

#686

#688

#689

#69

#690

#692

#693

#694

#695

#696

#697

#698

#699

#700

#704

#71

#710

#710

#715

#718

#72

#721

#722

#730

#734

#737

#76

#77

#78

#8

#88

#9

#9

#91

#93

#93

#97

#97

#98

0.99

1.0

Release_1.0

Release_1.01

Release_1.02

Release_1.03

Release_1.04

Release_1.1

Release_1.2

Release_1.3

Release_1.4

Release_1.5

Release_1.6

Release_1.68

Release_1.7

Release_1.72

Release_1.80

Release_1.82

Release_1.90

Release_1.92

de58043205 Added image license generation for solr image search results when results are generated within yjson result writer. This makes it possible to view images in yacyinteractive from solr. Michael Peter Christen 2013-02-13 00:33:53 +01:00
d3508fa8ff fixed json search, quotes, auto-facets, urls etc. for yacyinteractive.html Michael Peter Christen 2013-02-13 00:01:38 +01:00
1db23e9eac Moved methods from SolrServerConnector to AbstractSolrConnector with the result that most of these methods become superfluous in other classes. This is a generalization step towards multi-indexes in Solr. Michael Peter Christen 2013-02-12 22:03:10 +01:00
02fa31b5bf better filesearch layout Michael Peter Christen 2013-02-12 12:21:29 +01:00
e55ec3071d reduced number of facets in yacyinteractive (only filetype necessary) Michael Peter Christen 2013-02-12 12:00:54 +01:00
16d90859b7 reverted put-semantics back to as-usual in serverObjects and introduced an add-method to put in several objects for the same key Michael Peter Christen 2013-02-12 11:52:33 +01:00
0d888ff69e Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git Michael Peter Christen 2013-02-12 03:42:58 +01:00
c34af7fe94 extended JSON Response Writer and Opensearch Response Writer for the Solr search interface in such way that it is possible to use this interface for the yacyinteractive search. This search interface is now much faster using the Solr search directly. For the Solr interface it was necessary to create a translation from the YaCy search modifiers to the Solr facet selection. This was added in such a way that it becomes generic for the normal YaCy search and as a on-top evaluation for Solr queries. Michael Peter Christen 2013-02-12 03:42:46 +01:00
c37d718f16 make sure yacy.running is deleted if not running (catch exception) - to prevent following log if YaCy was previously not properly shutdown reger 2013-02-11 22:53:19 +01:00
762b687e47 extended the serverObjects to be able to hold multipel values for a single key. This is done using the solr class MultiMapSolrParams. That class is needed in the OpensearchResultWriter to get multiple facet requests. Michael Peter Christen 2013-02-11 22:12:15 +01:00
d70d99fab5 added more metadata fields and facets to OpensearchResponseWriter. This should make it possible to replace the original and enriched yacy opensearch result with a solr output in opensearch format. Michael Peter Christen 2013-02-11 22:10:14 +01:00
6a4878940b fix in html parser and bookmark generation Michael Peter Christen 2013-02-11 13:28:08 +01:00
51e7ab4f70 moved bookmarks back to more prominent location (even if this does not fit to the 'Search Interfaces' headline) Michael Peter Christen 2013-02-09 06:57:20 +01:00
dee8b24d3c better error handling for bookmarks Michael Peter Christen 2013-02-09 06:55:57 +01:00
e1da39245a when searching the network, do not search on robinson peers with the old DHT search interface. Now use the solr interface. Michael Peter Christen 2013-02-08 18:30:08 +01:00
6f6ddaf7e7 A robinson peer does not need to write RWI data if such peers are only searched using the solr interface. Searching public rpbinsons will be done with solr only in the future. Michael Peter Christen 2013-02-08 17:58:54 +01:00
ab4f74c82c fix for xml blacklist import Michael Peter Christen 2013-02-08 15:12:10 +01:00
7806680ab8 fixed a problem with re-feeding of already indexed documents whith coordinates attached. Michael Peter Christen 2013-02-08 12:45:54 +01:00
cb38e860cf After the observation that Windows user simply forget that they started YaCy; YaCy is still running and the user additionally expect that another doubleclick on the YaCy icon simply opens the search windows (again) I decided to add a function that complies to the expectation to the user: simply open the browser pop-up page again if the user starts YaCy while YaCy is still running. Michael Peter Christen 2013-02-07 23:39:00 +01:00
27894d2c1a Merge branch 'master' of git@gitorious.org:yacy/rc1.git Marc Nause 2013-02-05 21:09:41 +01:00
75f9568472 *) only install files from the RELEASE directory *) minor changes Marc Nause 2013-02-05 21:02:32 +01:00
eb80405a16 added a disable function in RemoteCrawl_p servlet which prevents setting of remote crawl if peer is not a senior or principal peer Michael Peter Christen 2013-02-05 12:47:20 +01:00
1e3d8cc235 show a link for the host in the host browser; see Michael Peter Christen 2013-02-04 21:24:57 +01:00
19c46e4acf catch more exceptions Michael Peter Christen 2013-02-04 21:24:39 +01:00
7de502f43d Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git Michael Peter Christen 2013-02-04 20:02:35 +01:00
3bc5ee6e3d *) added protection against CSRF in update download page (http://localhost:8090/ConfigUpdate_p.html?releaseinstall=../../test.txt&deleteRelease=Delete+Release does not work anymore) Marc Nause 2013-02-04 19:57:28 +01:00
4f270d89e2 another NPE Michael Peter Christen 2013-02-04 18:04:52 +01:00
921091c3a6 use thread-safe http connection manager for authenticated remote solr connections Michael Peter Christen 2013-02-04 17:48:04 +01:00
e8f7b85b98 fixes to internal RWI usage if RWI is switched off (NPE etc) Michael Peter Christen 2013-02-04 17:11:02 +01:00
3834829b37 bugfixes and more logging for solr connector Michael Peter Christen 2013-02-04 16:42:10 +01:00
bc00097cbf arrr... forgot the new library Michael Peter Christen 2013-02-04 12:02:37 +01:00
09a2b09c48 guava update Michael Peter Christen 2013-02-04 11:21:05 +01:00
80fe3d7860 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git Michael Peter Christen 2013-02-04 10:57:54 +01:00
4323621a76 update to Solr 4.1.0 Michael Peter Christen 2013-02-04 10:55:49 +01:00
639c114199 remove jetty from classpath - as it was moved last commit sixcooler 2013-02-03 23:24:19 +01:00
160ce568b3 move testing SolrServlet.main to test, making include of jetty*.jar in distribution and classpath obsolete reger 2013-02-03 22:32:38 +01:00
07a20e8253 removed unused import orbiter 2013-02-02 10:52:39 +01:00
d1cb4cbc84 enhanced network scanner, is faster and more flexible now - start more processes - remove superfluous host name resolution - better/more flexible subnet ip range calculation - prefer ipv4 makes better usable ip pre-settings in servlet - extended servlet by new subnet /20 - option - redesign of scanner start process in servlet (generalization) Michael Peter Christen 2013-02-02 09:51:43 +01:00
592adf7ccb fix for domain navigation Michael Peter Christen 2013-02-02 07:21:18 +01:00
4ca1b76627 less search overhead when first result set is smaller than requested Michael Peter Christen 2013-02-02 07:20:56 +01:00
f748b0aa7c NPE fix Michael Peter Christen 2013-02-02 07:20:02 +01:00
7dfcc92b71 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git Michael Peter Christen 2013-01-31 13:15:42 +01:00
0b6566a389 optimizations when starting large crawl requests with many start urls in one request: - allow larger match-fields in html interface - delete all host hashes at once from zurl - when deleting by host, do not count size of deleted entries since that was the reason it took so long Michael Peter Christen 2013-01-31 13:15:28 +01:00
a2160054d7 ability to create vocabularies also without any objectspace: this iterates over all urls in the index do create terms orbiter 2013-01-30 19:33:48 +01:00
ecc10a752c fixes to index enumeration for vocabulary production orbiter 2013-01-29 18:14:14 +01:00
be5d3a1066 adding classpath to Manfiest of yacycore.jar - this allows to start w/o giving explicite java -cp (just java -jar lib/yacycore.jar works) - especially helpful while running YaCy as Win service, making it obsolete to adjust classpath cfg of the service wrapper on upgrades of lib/*.jar's reger 2013-01-29 03:01:57 +01:00
be27567b53 allow more links when starting a crawl by file Michael Peter Christen 2013-01-28 17:50:23 +01:00
3777b338c7 bugfix: location url for migrate urldb button onclick reger 2013-01-27 06:13:49 +01:00
8447814a31 correct headermenue in migrateurldb_p.html - update NetBeans project path reger 2013-01-26 23:43:09 +01:00
99185d7048 one more fix for author_sxt Michael Peter Christen 2013-01-26 03:59:39 +01:00
b6ae6262f6 - add the copyField author_sxt only if author exists - set the solr default search field according to existing fields Michael Peter Christen 2013-01-26 03:34:46 +01:00
088373b4ea catch exception if solr connection change fails Michael Peter Christen 2013-01-25 16:06:58 +01:00
8a55fd96e9 Merge remote-tracking branch 'aleksejs/rutrans3' Michael Peter Christen 2013-01-25 13:57:10 +01:00
3252223055 Russian translation fixes and additions Aleksej 2013-01-25 16:05:48 +04:00
7293c0413b Russian localization:index.html fix Dmitriy Kazimirov 2013-01-25 16:08:08 +07:00
3a13906121 clear some more caches if running out of memory sixcooler 2013-01-25 04:24:36 +01:00
6fc4bdddbd *) fixed admin password configuration Marc Nause 2013-01-24 20:09:33 +01:00
e23a596c1d added a copyField for author_sxt for automated schema generation Michael Peter Christen 2013-01-24 18:25:28 +01:00
8651ec35fe turned author_s into the multi-valued field author_sxt Michael Peter Christen 2013-01-24 18:24:31 +01:00
f1a4feda3e security fix for suggest (don't let users ask for too much) Michael Peter Christen 2013-01-24 17:57:28 +01:00
244b157299 fix for external solr schema definition Michael Peter Christen 2013-01-24 16:34:15 +01:00
4589afe056 fix NPE when solr does not deliver snippets Michael Peter Christen 2013-01-24 14:12:31 +01:00
0fe7b6fd3b migrated the index export methods from the old metadata to solr. Now exports are done using solr queries. removed superfluous methods and servlets. Michael Peter Christen 2013-01-24 12:39:19 +01:00
1768c82010 removed field selection because that created documents with that field only which was not useful when re-writing the same document Michael Peter Christen 2013-01-24 03:26:38 +01:00
8eebeea533 fix for search result link in ViewFile Michael Peter Christen 2013-01-24 01:50:59 +01:00
5bed1a7893 Russian localization update Dmitriy Kazimirov 2013-01-23 17:11:45 +07:00
31e854bef6 Merge remote-tracking branch 'copro/master' Michael Peter Christen 2013-01-23 14:41:17 +01:00
4735bd47f4 - changed solr commit call and added an optimize option. Since Solr 4.0.0 there is a new softcommit feature which implements a near-real-time (NRT) search option. The softcommit does not do IO and does not cause performance issues. YaCy has now an extension in its solr connectors to use the softcommit feature. The softcommit call now replaces all places where a hard commit was used. Furthermore the commit strategy in when doing a search from the web interface was changed (it's done every time before a search is done). Michael Peter Christen 2013-01-23 14:40:58 +01:00
0025983993 Fix typo embedd -> embed Copro 2013-01-23 04:11:55 +01:00
3ea8380959 Adding Vimeo tag to wiki commands to embedd Video video with id Copro 2013-01-23 04:00:15 +01:00
ee9d7fd93d Added feature to embedd Youtube videos to wiki commands for usage in Wiki, Blog or other servlets Copro 2013-01-23 02:43:58 +01:00
ec927ea72b Merge remote-tracking branch 'reger/master' Michael Peter Christen 2013-01-22 17:01:49 +01:00
7159ed2a7d Merge remote-tracking branch 'copro/master' Michael Peter Christen 2013-01-22 17:01:18 +01:00
946fad48c7 Some more German translation reducing the amount of Unused String messages Copro 2013-01-22 15:33:49 +01:00
6690dac845 Russian translation fixes not merged due to conflict Aleksej 2013-01-22 16:19:07 +04:00
9ccdd21d76 Merge remote-tracking branch 'aleksejs/fixtrans' Michael Peter Christen 2013-01-22 11:54:38 +01:00
de7c3d95b4 Added German translation for HostBrowser.html Copro 2013-01-22 05:14:37 +01:00
5e5ae01909 updated Russian localization for update system Dmitriy Kazimirov 2013-01-21 21:32:12 +07:00
f9c65078f0 A little more fixes for Russian localization Dmitriy Kazimirov 2013-01-20 20:01:29 +07:00
ca01d225db A little more fixes for Russian localization Dmitriy Kazimirov 2013-01-20 17:26:11 +07:00
9dc0bea1dc Little more correct and readable Russian localization Dmitriy Kazimirov 2013-01-20 04:52:37 +07:00
c1b9113a68 Little more correct and readable Russian localization Dmitriy Kazimirov 2013-01-20 04:43:55 +07:00
9cc72df176 More Russian translations. And if some text is not translated it will be in English and not German Dmitriy Kazimirov 2013-01-19 20:21:47 +07:00
db024a4e19 added new solr fields (unused yet; implementation will follow) Michael Peter Christen 2013-01-21 18:02:29 +01:00
f5fd2aea18 removed archaic migration code Michael Peter Christen 2013-01-21 17:59:42 +01:00
9b5bdae1b4 Reverted setting of MMapDirectoryFactory from solrconfig; see http://forum.yacy-websuche.de/viewtopic.php?p=27509#p27509 Instead, in the start script is checked if the host is a 64 host and -Dsolr.directoryFactory=solr.MMapDirectoryFactory is set as java option Michael Peter Christen 2013-01-21 17:55:28 +01:00
f8f7f33596 add Maven build script reger 2013-01-20 21:08:59 +01:00
eb68a30947 solr performance settings the target of these performance settings is the reduction of IO in general and during search in particual. - reduced mergeFactor to 4. This will increase the IO during indexing, but will reduce IO during search. It will also greatly reduce the number of open files which should make it possible to have overall larger indexes until the number of open files in an OS is reached. - increased ramBufferSizeMB to 256mb. This will reduce the number of commits. This change may compensate the reduction of the mergeFactor. - disabled updateLog. This is a real-time search feature which is available in YaCy anyway because a commit is forced if index.html is called. The updateLog feature causes a lot of IO during indexing and search and produced a lot of files in SEGMENTS/solr_40/data/tlog orbiter 2013-01-19 11:21:33 +01:00
60f2a69331 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git Michael Peter Christen 2013-01-17 21:53:19 +01:00
cba038f97b one more NPE fix Michael Peter Christen 2013-01-17 21:52:56 +01:00
f3e705c4fe bump to httpclient / httpcore 4.2.3 (bugfix-release) sixcooler 2013-01-17 20:10:49 +01:00
aa067da86b set the 'all' option as option at end of the list because the all option currently select also lists which cannot be exported in xml correctly Michael Peter Christen 2013-01-17 01:04:50 +01:00
af465cdca5 fix for wrong robots.txt loading for https protocol see also: http://forum.yacy-websuche.de/viewtopic.php?f=5&t=4579 Michael Peter Christen 2013-01-16 17:38:06 +01:00
edbc86d2b0 integrated search term into opensearch result title. this makes better bookmark names when subscribing multiple search results from the same peer Michael Peter Christen 2013-01-16 16:18:03 +01:00
c3d50d91f8 relaxing site operator for www prefix: - when using a site operator search for a domain where the domain has a www prefix, also the domain without the www is enclosed - when using a site operator search for a domain where the domain has no www prefix, also the domain with the www in enclosed - in the host navigator, all domains with and without a www prefix are accumulated. That means that the host navigator does never show a host with a www prefix. This should prevent usage mistakes of the site operator. Michael Peter Christen 2013-01-16 14:54:35 +01:00
f53703df62 using MMapDirectoryFactory as solution for ClosedChannelException given in https://issues.apache.org/jira/browse/SOLR-2247 Michael Peter Christen 2013-01-16 14:35:37 +01:00
db49e91724 fixed a NPE which may appear for freeworld peers without any rwi index data. This the NPE looked like: Caused by: java.lang.NullPointerException at net.yacy.search.query.SearchEvent.<init>(SearchEvent.java:279) at net.yacy.search.query.SearchEventCache.getEvent(SearchEventCache.java:155) at search.respond(search.java:314) ... 12 more Michael Peter Christen 2013-01-16 11:07:20 +01:00
4faa07c214 added a timeout for topic computation (solr is here much slower than the old metadata-db) Michael Peter Christen 2013-01-15 16:20:43 +01:00
d2d5be032d added a 'inlink' search option according to the suggestion in the YaCy forum at http://forum.yacy-websuche.de/viewtopic.php?f=18&t=4572#p27410 Michael Peter Christen 2013-01-14 12:50:21 +01:00
76e1e91b11 with strict compiler settings, IndexFederated_p does not compile without @SuppressWarnings("deprecation") Michael Peter Christen 2013-01-14 12:33:01 +01:00