Commit Graph

  • 777cb5b812 remove test case for Standard_MemoryControl which will always fail see https://github.com/yacy/yacy_search_server/pull/114 reger 2017-04-02 03:59:37 +02:00
  • ba339a2a45 Add servlet to import warc file from filesystem IndexImportWarc_p.html. Apply Importer interface to WarcImporter reger 2017-04-02 03:32:21 +02:00
  • 1d81b8f102 Merge branch 'master' of git@github.com:yacy/yacy_search_server.git Michael Peter Christen 2017-04-01 01:04:27 +02:00
  • 69081bce00 added export to elasticsearch. The export dump can easily be imported to elasticsearch using the command curl -XPOST localhost:9200/collection1/yacy/_bulk --data-binary @yacy_dump_XXX.flatjson Michael Peter Christen 2017-04-01 01:04:17 +02:00
  • 510f11d374 Implement surrogate import from Warc archives (as first option handle warc = Web ARChive File Format. Warc files with extension .warc or compressed warc.gz can be placed in the DATA/surrogate/in and contained responses are imported to the index. The used library is stream based so we can easily extend it later to use and load warc's from the net. reger 2017-03-31 00:58:11 +02:00
  • 5b5b9d5d96 URL Viewer : only display the link to metadata when metadata exists luccioman 2017-03-30 16:14:22 +02:00
  • 4b649b0a11 Fixed NPE case and API URL link on Solr HTML output for webgraph core. luccioman 2017-03-30 15:41:14 +02:00
  • 39ffa42a3c Modified RWI settings page radio click event to use HTTP POST luccioman 2017-03-30 10:23:47 +02:00
  • af28a07780 Updated API calls recording/replay with recent changes. luccioman 2017-03-30 09:22:28 +02:00
  • 1ccc44e681 fix default/httpd.mime Z file extension to lower case + test case reger 2017-03-26 23:52:31 +02:00
  • 44a9a580e3 remove seedlist bootstrap target (not working for some longer time) reger 2017-03-26 23:26:40 +02:00
  • c16498305b Add label text for search word statistic (AccessTracker_p.html) to master lng file reger 2017-03-26 23:13:12 +02:00
  • 81670c3484 One more use of SwitchboardConstants.SERVER_PORT constant, apply standard servlet design pattern initialization of solrselectservlet reger 2017-03-26 20:05:48 +02:00
  • cde237b687 Enforced access controls on some administrative actions. luccioman 2017-03-26 11:48:00 +02:00
  • df5970df6d Extended Apache HTTP Digest Auth. for use of YaCy encoded password luccioman 2017-03-26 11:29:04 +02:00
  • 40403942db Updated dump/restore shell scripts : the API is now IndexExport_p.html luccioman 2017-03-26 10:59:04 +02:00
  • 29e5110627 Updated shell scripts to be compatible with HTTP Digest authentication luccioman 2017-03-21 17:15:01 +01:00
  • bdadbda5fa Update master lng file with added text in Settings_ServerAccess remove outdated file entry in fr.lng & sk.lng reger 2017-03-21 01:16:16 +01:00
  • 1537157839 adjusted .travis.yml to build in libbuild first (see http://mantis.tokeek.de/view.php?id=545); added test of build instructions Karl-Philipp Richter 2016-10-25 05:06:42 +02:00
  • c55d526cb8 Add hint how to build with maven (for the first time) to readme reger 2017-03-20 02:33:21 +01:00
  • cbf58d5f0a Add hint text to default ServerAcess Port Settings page reger 2017-03-19 21:45:33 +01:00
  • f05976c017 Display the local search word statistic in alphabetic order reger 2017-03-19 07:12:35 +01:00
  • 3dd23c178b Introduce the option to configure a shutdown port. A port value of -1 will disable this option. reger 2017-03-19 02:30:08 +01:00
  • c4d5f1fc54 upd to slf4j-1.7.24.jar reger 2017-03-18 20:32:53 +01:00
  • c4b90eae98 upd to icu4j-58_2.jar reger 2017-03-18 20:06:58 +01:00
  • a2afb4bae0 add switchboardconstants for server ports config keys reger 2017-03-18 20:02:26 +01:00
  • e0c5b28331 update to jsoup-1.10.2.jar reger 2017-03-17 02:19:33 +01:00
  • 5b5ada38c3 update to jsch-0.1.54.jar reger 2017-03-17 02:07:02 +01:00
  • 038b9cd98e update translation for ConfigNetwork_p.html reger 2017-03-15 22:36:53 +01:00
  • f7fce1baad make digest default authentication in defaults/web.xml reger 2017-03-15 01:39:15 +01:00
  • 56d0a87a83 remove double occuance of geo:lat in rss tokens reger 2017-03-13 03:08:44 +01:00
  • 882d99dae4 upd to metadata-extractor-2.10.1.jar reger 2017-03-13 00:34:40 +01:00
  • b4fa1141b8 implement RequestHeader getRequestURI, getRequestURL for legacy request reger 2017-03-12 01:54:56 +01:00
  • 209a7374bd remove unused import pdfParser reger 2017-03-09 22:57:51 +01:00
  • de1c1c16db Improve pdf text extraction resource handling. For sort pdf <= 3 pages use already extracted content, only for long pdf > 3 pages reassign content and close internal writer (to direct free buffers) reger 2017-03-09 22:56:33 +01:00
  • 52c9d0c858 upd to pdfbox-2.0.4.jar reger 2017-03-09 22:50:19 +01:00
  • 9b6d1abd9e eliminate some compiler unchecked and deprecation warnings in nav plugins by explicite type declaration and replacing date.getYear with Calendar.get reger 2017-03-09 01:42:36 +01:00
  • 6eb7d27449 upd to httpclient v4.5.3 reger 2017-03-08 22:35:48 +01:00
  • 8e77fe3860 Fixed unresolved pattern case in search results progress bar. luccioman 2017-03-08 10:27:18 +01:00
  • 79df5bb20a Fixed settingsAck_p.html back link for case where referrer is stripped. luccioman 2017-03-07 12:27:27 +01:00
  • 18c7563dbe Extend DCEntry.getLanguage convert to ISO639-1 codes for more languages by using icu.ULocale for languages not already covered (ICU normalizes to ISO639-1 2 char codes). Add test class Use DublinCore vocabulary declarations in DCEntry and SurrogateReader for easier usage debugging, Init SurrogateReader.inputSource on first use. reger 2017-03-05 02:26:10 +01:00
  • ce87025462 further avoid to set connect info properties as header value following comment "use of properties as header values is discouraged" in case where (proxy)HTTPClient overwrites values with supplied url. Use defined request.referer procedure in response class. reger 2017-03-04 22:45:17 +01:00
  • cd4d891ea4 use pre-defined "Connection" header key, replace depreceated reger 2017-03-04 19:41:31 +01:00
  • 5b03feb776 Fixed unresolved pattern case on /yacysearchlatestinfo.json api luccioman 2017-03-03 13:46:44 +01:00
  • 0173b0bc32 Added an advanced settings page for referrer policy settings. luccioman 2017-03-03 12:05:30 +01:00
  • 81963a89fe fix proxyservlet response url to respect http scheme if a relative Location header is returned. reger 2017-03-03 00:21:56 +01:00
  • 9d9f86dcdd Updated Archive-It heuristics URL. luccioman 2017-03-01 09:43:00 +01:00
  • cdcd923375 Privacy enhancement : added settings to control referrer policy. luccioman 2017-02-28 18:11:54 +01:00
  • 86534a56f7 fixed ReindexSolrBusyThread new and unexpected repeat of same query with low number of found documents - by adding additional end condition to remove processed query with number of found docs <= process-chunck-size. reger 2017-02-27 23:00:46 +01:00
  • 0aa0dd0b5b fix delta time calculation in PerformanceSearch_p for the 1. entry (INITIALIZATION displayed absolute date, set delta to 0 for 1. entry) reger 2017-02-27 01:04:31 +01:00
  • 13c5c09518 Fixed datacite.org heuristics base url. luccioman 2017-02-26 11:03:15 +01:00
  • 275c0cddd1 Adjust DefaultServlet test case to recent change, depreciate unused CONNECTION_PROP_PROTOCOL (also as it might be misleading with getProtocol vs getScheme) reger 2017-02-26 02:39:52 +01:00
  • 41e2ee0eca Fix call parameter for ConnectionInfo in MonitorHandler (expected scheme e.g. http, was protocol version). Depreceate obsolete custom X-...-Scheme header constant. Use existing FORMAT_ANSIC Dateformatter in HeaderFramework. Correct htmlParserTest (del one not intended println) reger 2017-02-25 23:55:17 +01:00
  • 9e626f6b00 Added a hint title for required fields in the Solr Schema editor luccioman 2017-02-24 11:09:42 +01:00
  • ac766327d3 Switched a few more Solr fields from strictly mandatory to optional luccioman 2017-02-24 11:08:18 +01:00
  • f254fcfc67 fix htmlParser <script> text extraction on code containing expression recognized as tag like 1<a reported in https://github.com/yacy/yacy_search_server/issues/109 reger 2017-02-24 01:25:32 +01:00
  • 2f191e0e1c Improved MultiprocotolURL non ASCII characters support. luccioman 2017-02-23 11:09:43 +01:00
  • 18e8b3a220 Merge branch 'escape' of https://github.com/sinkuu/yacy_search_server luccioman 2017-02-23 11:03:05 +01:00
  • 562fc14eb9 Merge pull request #110 from goofy-bz/patch-1 luccioman 2017-02-23 07:52:55 +01:00
  • 72a1bc0af1 Fixing some typos goofy-bz 2017-02-23 01:13:31 +01:00
  • 7419989de3 Correct dublincore title property text to lowercase in htmlresponsewriter, remove unused (carry over) local variable Do the same for other responsewriter. reger 2017-02-23 00:27:56 +01:00
  • 4fdc11cae8 Update SearchEvent.java Burkhard 2017-02-22 02:01:48 +01:00
  • cdc7f3e431 Switched some Solr fields from mandatory to optional luccioman 2017-02-21 22:59:11 +01:00
  • 7c188ad092 Add extract of queries.log in form of top search word cloud (last 7 days) to AccessTracker_p.html (Network Access -> Local Search Log page). It displays top 20 words of search queries. reger 2017-02-20 23:27:33 +01:00
  • 3475d8c1a9 Merge branch 'master' of https://github.com/yacy/yacy_search_server.git luccioman 2017-02-20 10:48:44 +01:00
  • c68a8be2d9 Refactored and enforced Solr mandatory fields for proper operation luccioman 2017-02-20 10:48:07 +01:00
  • 334c70c37a correct fromDate init value on missing param in api/timeline_p servlet revert test modification from last commit in AccessTracker.main reger 2017-02-20 00:14:14 +01:00
  • cc770512d5 add hint of query syntax in AccessTracker log (qs=normal querystring, sq=solr-querystring) to allow to filter simple text queries for processing, remove toString for counter parameter use more predefined constants in solrservlet reger 2017-02-19 05:23:17 +01:00
  • e5858bc8c8 Fixed a NullPointerException case possible on Index Export luccioman 2017-02-17 11:09:30 +01:00
  • 7e53860fc7 fix NPE in HTMLResponseWriter on missing document title reger 2017-02-16 02:36:24 +01:00
  • 5e8879beb7 Reduce self generated content for text_t (visible text index field) to avoid repeat of tokenized url as description, continuation of 7e09bff4a1 1409cabe8b Add some javadoc, and not needed remove of omitted fields in postprocessing. reger 2017-02-16 01:43:14 +01:00
  • 6ec6ab55ba removed faroo news from default opensearch config As @luccioman informed, it's only useable with a free api key http://www.faroo.com/hp/api/api.html http://blog.faroo.com/2013/06/30/faroo-introduces-an-api-key/ reger 2017-02-15 23:26:54 +01:00
  • 6e89d125f2 Added robots.txt support for heuristics federated search. luccioman 2017-02-15 15:04:40 +01:00
  • a46b232bf1 Use java.net.URLDecoder sinkuu 2017-02-11 16:10:14 +09:00
  • 7e6e14a406 adjust translation to renamed configparser_p.html reger 2017-02-14 02:30:26 +01:00
  • a011a97de9 make ConfigParser a protected page, for consistent behavior of locked menu items. reger 2017-02-14 02:04:42 +01:00
  • f85aaa7c76 update opensearch conf - remove suche.sueddeutsche.de apparently they've revoked the participation in opensearch initiative. reger 2017-02-14 00:31:32 +01:00
  • bf16de29c1 Added support for HTML OpenSearch results. luccioman 2017-02-13 19:11:17 +01:00
  • a79194a102 upd to Jetty-9.2.21.v20170120 reger 2017-02-11 19:53:27 +01:00
  • 4306f4d9a3 Upgraded Apache Ant to 1.10.1 in the Docker alpine flavor image luccioman 2017-02-10 09:40:42 +01:00
  • 54405577aa Replaced absolute redirection locations by relative ones when possible. luccioman 2017-02-09 16:42:21 +01:00
  • 1857651988 Added a new Debug/Analysis advanced settings subsection. luccioman 2017-02-09 11:05:06 +01:00
  • 526f2d6a8b Fixed NPE case occurring when local solr index is disabled in search. luccioman 2017-02-09 10:59:41 +01:00
  • def55ec166 Improved termination of timed out remote solr requests to peers. luccioman 2017-02-06 12:41:24 +01:00
  • 94af489f14 Removed deprecated "localMissCount" prop from yacysearchlatestinfo.json. luccioman 2017-02-03 10:32:31 +01:00
  • 08de58b6d3 Named a Thread without name for easier monitoring luccioman 2017-02-03 09:55:08 +01:00
  • 9a5a124bf2 Distinguished solr connectors thread names for easier monitoring. luccioman 2017-02-03 09:54:29 +01:00
  • f6ad927a14 Refactored the DHT-Trigger section in Performance_p.html page. luccioman 2017-02-01 18:44:42 +01:00
  • 85d8173b1e Updated French translation for the /Performance_p.html page. luccioman 2017-01-31 16:33:17 +01:00
  • b51fd9467c Fixed unresolved pattern on directory entries in HostBrowser.xml api. luccioman 2017-01-31 09:20:19 +01:00
  • f6b08443f0 adjust column layout in Settings_Proxy.inc reger 2017-01-30 22:44:28 +01:00
  • 21ab41d8d6 Merge branch 'master' of https://github.com/yacy/yacy_search_server.git luccioman 2017-01-28 10:20:25 +01:00
  • 95b63f5126 Added a CSS class for infobox block. luccioman 2017-01-28 10:19:39 +01:00
  • 8ce8e23e7d Update language file de & master, remove obsolete "Augmented Browsing" reger 2017-01-28 01:13:57 +01:00
  • 1f497ccad5 Add consistency check for related index fields upon load and save of index schema. To assemble the original link url for out-/inboundlinks, icons and pictures the *_protocol_sxt and *_urlstub_sxt is needed (due to the used data-reduced storage methode). Auto-enable *_protocol_sxt if *_urlstub_sxt is enabled. to be able to correctly assemble the original link url. reger 2017-01-28 00:36:03 +01:00
  • 68afe900d0 Added user-friendly controls over disk usage configuration settings. luccioman 2017-01-27 15:47:15 +01:00
  • 95d2a28599 adjust the Field-Reindex Thread to verify and update the document id in case hash (ID) doesn't match document url (sku field). reger 2017-01-26 23:49:15 +01:00
  • e6e4ccaa00 Merge pull request #98 from Velociraptor85/patch-2 Michael Christen 2017-01-26 06:37:29 +01:00
  • a7fd47b3aa Merge pull request #105 from ivar/patch-1 Michael Christen 2017-01-26 06:29:42 +01:00
  • cfd21aaa10 Update README.md - removes deprecated URL Ivar Vasara 2017-01-25 20:36:48 -08:00