b1ace63607
codespell: spelling corrections
2021-05-06 01:52:55 +10:00
1bc5fecb33
website updates
2014-08-31 11:11:12 -07:00
8149e99965
added developer.html warning msg.
2014-05-26 11:41:12 -06:00
8bdb9d1a3e
doc updates per john on how we dedup
2014-01-30 10:57:49 -08:00
8876dae984
added and fixed support for <link ahref=xxx rel=canonical>.
...
treat those as simplified meta redirects.
updated spider dedup documentation in developer.html file.
2014-01-30 10:37:59 -08:00
6a45e42128
added ability to treat <link xyz.com rel=canoical> as meta redirects.
...
should help us dedup.
added a function to do looser deduping of spider pages although current
not enabled, we are still using the more strict one.
added documentation on how we dedup to developer.html for jon to
take a look at.
2014-01-30 10:04:09 -08:00
970d5b2488
formatting
2014-01-19 16:40:22 -08:00
47465f6d90
more fixes. trying to fix spiders to
...
spider multiple urls from same ip...
2013-09-19 11:13:40 -07:00
6332de2daf
added link to compare.html comparison to SOLR
...
into documentation.
2013-08-21 13:14:17 -06:00
37a6549a58
updates to developer.html developer
...
documentation. removed a lot of obsolete
information. still needs more work.
2013-08-21 13:09:55 -06:00
8971d9b932
comment our urldb from developer.html
...
since no longer used.
2013-08-21 08:59:51 -06:00
6cf0497c2c
added a little posdb documentation to
...
developer.html. posdb replaced indexdb
as the new index because it has word
position info as well as word field info.
2013-08-21 08:40:28 -06:00
6f64568ef8
A bit of html cleanup and added <pre> style.
2013-08-05 10:31:52 -06:00
b002233b02
Updated some of the content and put some comments about possibly removing internal gigablast notes.
2013-08-05 10:17:41 -06:00
f6e560c1f4
Initial file population.
2013-08-02 13:12:24 -07:00
