2e7412abbd
Use rwlock for RdbBuckets
2017-06-06 17:15:48 +02:00
3c621c4f31
Add debug-test as target
2017-05-10 16:25:46 +02:00
1d769203e7
Merge branch 'nomerge2'
2017-05-08 11:59:43 +02:00
1c117d3aa1
Dropped PingInfo + PingServer entierely.
...
The remaining functionality of the pingserver was to ping normalt and "shotgun" IPs to determine the fastests and working network path. This is better done at L2/L3 or OS level - not in the application.
The email-sending functionality was dropped too. We recommend using 3rd-party tools for monitoring and statistics-collection/triggering.
2017-05-05 15:00:31 +02:00
61dc9e4573
Keep our own Host::m_flags up-to-date
2017-05-04 14:37:19 +02:00
d3ecbda478
Use Vagus for some host/instnace information
...
hosts.conf CRC and total_docs_indexed is now exchanged over Vagus. rest will follow in separate commits.
The vagus cluster identifier defaults to "gb-"$USER
2017-05-01 17:21:34 +02:00
97a480a0f6
Merge branch 'master' into nomerge2
2017-04-18 14:12:38 +02:00
53e89c2d15
Revert watchdog functionality.
...
Reverting commit:
00d91dc2e3
a57aa86bb6
The watchdog functionality worked fine but added a non-neglible background load on idle setups and scaled just as poorly as the pinginfo. We need a better mechanism.
2017-04-18 14:10:45 +02:00
211b210ccb
Merge branch 'master' into nomerge2
2017-04-10 13:51:36 +02:00
c86745045d
Removed Msg1f / remote log fetching
...
Program logs too much for last 20KB log of being any use. Viewing log in a web interface is better done with a 3rd-party log thing (logstash/syslog-ng/fluentd/...)
2017-04-10 13:47:33 +02:00
50b656f93a
Merge branch 'master' into nomerge2
2017-04-10 12:38:13 +02:00
766facf75d
Remove blaster/blasterdiff command that seems to be processing gb output log and injecting url. We should do that outside of gb and use the admin/inject interface if needed.
2017-04-06 19:16:39 +02:00
a57aa86bb6
Send watchdog messge to hosts we haven't interacted with recently.
...
Distinquish between hotss we normalyl talk to and hosts we rarely talk to. Use agressive conditional watchdog messages, and detect unresponsive hosts based on that.
2017-04-06 17:18:56 +02:00
813c4e2350
Remove now unused Timezone.cpp
2017-04-06 14:45:48 +02:00
8ae62f5e22
Move SpiderCache to separate file.
...
Move spider page related code to PageSpider.cpp
2017-04-05 16:13:51 +02:00
524cd5ad85
Remove now unused Msg1
2017-04-04 16:22:54 +02:00
68ac6ed451
Add debug-safe, debug-test config target
2017-04-03 20:55:24 +02:00
75db44d81d
Merge branch 'master' into nomerge2
2017-03-27 15:05:27 +02:00
7f3a5f4cb8
Re-add warning suppression
2017-03-27 15:05:10 +02:00
7a21587ca6
Merge branch 'master' into nomerge2
2017-03-27 15:03:19 +02:00
dad3e273b8
Move clang++ if to before g++. clang++ matches g++ as well.
2017-03-27 14:54:16 +02:00
213cb460cd
Merge branch 'master' into nomerge2
2017-03-22 20:46:44 +01:00
1778d66315
Remove statsdb
2017-03-22 20:46:28 +01:00
f27190e251
Merge branch 'master' into nomerge2
2017-03-17 15:12:58 +01:00
98192ba79d
Move generateGlobalIndex to thread
2017-03-17 14:58:26 +01:00
d9b47779cc
Merge
2017-03-16 17:21:44 +01:00
e90e1d210b
Split Msg4.* into Msg4In and Msg4Out
2017-03-16 16:04:27 +01:00
aebc2e8ff8
Cater for multiple criteria types for UrlBlockList to speed things up
2017-03-06 16:12:43 +01:00
56915e22ea
Cater for multiple criteria types for UrlBlockList to speed things up
2017-03-01 15:37:31 +01:00
ac875d9e15
Merge branch 'master' into nomerge2
2017-02-09 12:19:55 +01:00
1def0b382a
Add thin wrapper around pcre.h
2017-02-07 16:38:54 +01:00
484d4e6ae7
Added real-time URL classification (not used yet from msg40)
2017-02-07 16:22:49 +01:00
adc20e3129
Fix timeing bug in parallele make of Version.o
2017-02-07 16:00:03 +01:00
728046f472
Make sure warning suppression work with custom version of g++ (eg: on travis)
2017-02-06 15:44:27 +01:00
5c329eed42
Fix makefile to allow for custom CXX in version check
2017-02-06 15:26:27 +01:00
538c75b205
Merge branch 'master' into nomerge2
...
Conflicts:
Parms.cpp
main.cpp
2017-01-25 20:29:30 +01:00
d725ded930
Refactor scoring weights into a struct
2017-01-20 16:27:16 +01:00
8fea89b797
Add url block list
2017-01-18 17:17:33 +01:00
55a91ea697
Merge branch 'master' into nomerge2
...
Conflicts:
Makefile
Msg2.cpp
Msg40.cpp
2017-01-09 14:25:37 +01:00
91c24650f5
Moved gbcompress/gbuncompress from XmlDoc.* to separate file
2017-01-09 14:00:34 +01:00
7cb999330b
Merge branch 'master' into nomerge2
...
Conflicts:
PosdbTable.cpp
2017-01-06 17:11:24 +01:00
d541968da7
Load docid2flagsandsitemap.dat
2017-01-06 17:06:33 +01:00
e003f35eab
Merge branch 'master' into nomerge2
2016-12-19 15:01:31 +01:00
08b20fddce
Prep. for page temperature in ranking
2016-12-19 12:24:44 +01:00
5a06b25687
Merge branch 'master' into nomerge2
...
Conflicts:
Msg2.cpp
Msg3.cpp
Msg39.cpp
Msg51.cpp
RdbList.cpp
2016-11-03 17:36:50 +01:00
ca0f5d52e6
Disable gcc suggestion warnings.
2016-11-01 12:35:16 +01:00
9e07acbce9
changed Makefile so config=release-safe must be used to disable features unsafe for our production environment
2016-10-31 20:54:21 +01:00
5bad84aa52
Merge branch 'master' into nomerge2
...
Conflicts:
Msg3.cpp
Msg39.cpp
Rdb.cpp
RdbCache.h
XmlDoc.cpp
2016-10-31 11:31:03 +01:00
9454e3df02
Added moveFile2Phase1()/moveFile2Phase2()
2016-10-24 15:15:35 +02:00
a93e34e085
Added makePath() function
2016-10-21 13:21:49 +02:00