Forks/privacore-open-source-search-engine
1
0
Fork 0
forked from Mirrors/privacore-open-source-search-engine
Code Pull Requests Activity
6,500 Commits 7 Branches 0 Tags
feature-summaryxpath
Go to file
Clone
Open with VS Code Open with VSCodium Open with Intellij IDEA
Download ZIP Download TAR.GZ Download BUNDLE
Ai Lin Chia 4a6e93fc78 Add configurable xpath for summary
2016-09-01 13:43:07 +02:00
antiword-dir
…
doxygen
Change doxygen setting
2015-12-07 23:47:28 +01:00
html
Remove EURLHASNOIP. g_errno is never set to EURLHASNOIP
2016-06-29 16:39:51 +02:00
misc
Log function will now return void instead of a boolean
2016-08-01 18:12:10 +02:00
script
Merged 6e12f96aeab29778b3b6ba64c5fe594e7e6989cd
2015-11-20 12:01:09 +01:00
test
Add configurable xpath for summary
2016-09-01 13:43:07 +02:00
third-party
Add slacktee as third-party git submodule
2016-07-20 18:16:22 +02:00
tools
Use RdbQueryIndex and removed functions replaced by RdbQueryIndex
2016-08-31 11:13:31 +02:00
ucdata
Remove unused unicode "combiningclass"
2015-11-26 17:49:25 +01:00
.clang-format
Add clang format file (initial edition)
2015-12-22 10:30:57 +01:00
.gitignore
Modify header dependency method
2016-08-26 15:26:06 +02:00
.gitmodules
Add slacktee as third-party git submodule
2016-07-20 18:16:22 +02:00
.valgrindrc
Add valgrind rc file for default configuration
2016-05-19 11:00:04 +02:00
Abbreviations.cpp
Make abbreviation list thread-safe
2016-08-19 11:32:34 +02:00
Abbreviations.h
Standardize header guards
2016-03-08 22:16:02 +01:00
AdultCheck.cpp
Some log changes to pass in specific log levels
2016-08-05 10:05:56 +02:00
AdultCheck.h
Refactored XmlDEoc's adult-check to a separate file
2016-05-09 15:51:11 +02:00
antiword
fix calls to antiword and pdftohtml etc.
2014-06-15 17:44:52 -07:00
BigFile.cpp
Remove unused variable from BigFile::open
2016-08-31 11:13:31 +02:00
BigFile.h
Remove unused variable from BigFile::open
2016-08-31 11:13:31 +02:00
BitOperations.h
Enhanced debug output for RdbList/Posdb merging. Removed unused code
2016-04-06 16:42:31 +02:00
Bits.cpp
Fix s_init check in Bits.cpp. It should be a static
2016-07-22 13:59:55 +02:00
Bits.h
More constness in Bits.*
2016-05-13 17:11:59 +02:00
Blaster.cpp
Add constness to some tld/domain/url functions
2016-08-11 12:58:51 +02:00
Blaster.h
Moved local structs from Blaster.cpp to Blaster.h
2016-08-09 13:47:25 +02:00
bmptopnm
…
Clusterdb.cpp
Remove always true dedup from Rdb::init
2016-08-31 11:13:31 +02:00
Clusterdb.h
Remove extra semicolons
2016-05-19 18:37:26 +02:00
Collectiondb.cpp
#include cleanup in Repair.*
2016-08-31 19:22:52 +02:00
Collectiondb.h
Use rdbid_t in Collectiondb.*
2016-08-25 22:41:11 +02:00
Conf.cpp
Remove redundant code
2016-08-31 11:13:31 +02:00
Conf.h
Remove now unused g_conf.m_statsdbMaxCacheMem
2016-08-31 11:13:31 +02:00
control.deb
package bldg updates
2014-06-16 21:50:32 -06:00
copyright.head
package bldg updates
2014-06-16 21:50:32 -06:00
copyright.tail
package bldg updates
2014-06-16 21:50:32 -06:00
CountryCode.cpp
constness on static
2016-08-21 17:26:09 +02:00
CountryCode.h
Fix conversion from string literal to 'char *' for CountryCode
2016-05-31 11:26:00 +02:00
DailyMerge.cpp
#include cleanup in Repair.*
2016-08-31 19:22:52 +02:00
DailyMerge.h
Remove extra semicolons
2016-05-19 18:37:26 +02:00
Dir.cpp
Cleanup headers for Dir.h. Remove duplicated function to return dir filename
2016-08-31 11:13:31 +02:00
Dir.h
Cleanup headers for Dir.h. Remove duplicated function to return dir filename
2016-08-31 11:13:31 +02:00
Dns.cpp
(flexelint) change 'static const char *' in to 'static const char []'
2016-08-21 17:25:47 +02:00
Dns.h
Code style changes. More changes of hardcoded msg_type_t value to enum. Log levels.
2016-07-25 15:13:03 +02:00
DnsProtocol.h
Standardize header guards
2016-03-08 22:16:02 +01:00
Doledb.cpp
Remove always true dedup from Rdb::init
2016-08-31 11:13:31 +02:00
Doledb.h
Include RdbCache.h only in files where needed. Use fwd-decl elsewhere.
2016-07-18 16:22:06 +02:00
Domains.cpp
Add constness to some tld/domain/url functions
2016-08-11 12:58:51 +02:00
Domains.h
Add constness to some tld/domain/url functions
2016-08-11 12:58:51 +02:00
Entities.cpp
temporary fix for conversion of html entities to utf8, where the size of the utf8 string is larger than the length of the html entity name. would cause crashes in htmlDecode for e.g. ≪⃒
2016-07-13 17:16:10 +02:00
Entities.h
temporary fix for conversion of html entities to utf8, where the size of the utf8 string is larger than the length of the html entity name. would cause crashes in htmlDecode for e.g. ≪⃒
2016-07-13 17:16:10 +02:00
entities.json
entities.* fixes
2016-06-09 13:43:43 +02:00
Errno.cpp
Removed ENOTHREADSLOTS
2016-08-02 10:59:53 +02:00
Errno.h
Removed ENOTHREADSLOTS
2016-08-02 10:59:53 +02:00
fctypes.cpp
properly warn/print when clock is not in sycn with host 0
2016-08-19 11:44:10 +02:00
fctypes.h
Removed unused atod2()
2016-08-16 00:34:20 +02:00
File.cpp
constness on static
2016-08-21 17:26:09 +02:00
File.h
#include cleanup in File.h (and resulting other cleanup)
2016-08-10 00:32:00 +02:00
gb-1.0.spec
make it so we don't need --nodeps with
2014-05-25 22:08:46 -04:00
gb-include.h
First batch of changes streamlining emergency shutdown code
2016-06-20 12:30:26 +02:00
gb.deb.rules
if netpbm pkg already installed use it.
2014-07-06 09:54:28 -07:00
gb.pem
…
gbconvert.sh
Use /usr/bin/env instead of /bin/env which doesn't exist on ubuntu
2015-12-10 22:18:01 +01:00
GbFormat.h
Moved format #define out from HttpRequest.h so more code doesn't need to know the innaers of HttpRequest
2016-08-31 19:14:16 +02:00
GbMutex.cpp
Added GbMutex class
2016-08-15 16:24:56 +02:00
GbMutex.h
Added GbMutex class
2016-08-15 16:24:56 +02:00
gbstart.sh
Don't restart gb if uptime is less than 5 minutes. Use attachment for slack notifications.
2016-07-20 18:16:22 +02:00
generate_entities.py
entities.* fixes
2016-06-09 13:43:43 +02:00
giftopnm
…
GigablastRequest.h
Simplify Parms.h
2016-08-05 14:42:28 +02:00
hash.cpp
#include cleanup
2016-07-11 13:53:44 +02:00
hash.h
Made declaration and definition of inline hash functions consistent by removing the forward-decls
2016-08-23 10:22:45 +02:00
HashTable.cpp
Use gbshutdownAbortXxxx instead of g_process.shutdownAbort in HashTableT
2016-08-10 00:21:11 +02:00
HashTable.h
Remove extra semicolons
2016-05-19 18:37:26 +02:00
HashTableT.cpp
Removed HT_BUF_SIZE and commented-out referenecs to m_buf1/m_buf2
2016-08-16 00:39:09 +02:00
HashTableT.h
Removed HT_BUF_SIZE and commented-out referenecs to m_buf1/m_buf2
2016-08-16 00:39:09 +02:00
HashTableX.cpp
Some log changes to pass in specific log levels
2016-08-05 10:05:56 +02:00
HashTableX.h
Help flexelint
2016-08-21 17:18:11 +02:00
HighFrequencyTermShortcuts.cpp
bugfix highfreqfile: if last term had 0 entries it would fail
2016-05-23 12:32:22 +02:00
HighFrequencyTermShortcuts.h
Standardize header guards
2016-03-08 22:16:02 +01:00
Highlight.cpp
(flexelint) change 'static const char *' in to 'static const char []'
2016-08-21 17:25:47 +02:00
Highlight.h
#include cleanup of Highlight.*
2016-08-09 21:09:25 +02:00
Hostdb.cpp
Use rdbid_t in Hostdb.*
2016-08-25 00:29:40 +02:00
Hostdb.h
Use rdbid_t in Hostdb.*
2016-08-25 00:30:49 +02:00
HttpMime.cpp
More thread safety: gmtime() -> gmtime_r(), asctime() -> asctime_r()
2016-08-09 16:44:39 +02:00
HttpMime.h
more constness in HttpMime
2016-05-22 15:04:59 +02:00
HttpRequest.cpp
Minor #include cleanup in Collectiondb.h
2016-08-10 13:04:24 +02:00
HttpRequest.h
Moved format #define out from HttpRequest.h so more code doesn't need to know the innaers of HttpRequest
2016-08-31 19:14:16 +02:00
HttpServer.cpp
Removed unused parameter 'silten' from Loop::unregister*Callback()
2016-08-28 13:43:42 +02:00
HttpServer.h
#include cleanup of HttpServer.*
2016-08-10 18:18:49 +02:00
iana_charset.cpp
Turn while(1) loops into more conventional for(;;)
2016-08-21 17:35:23 +02:00
iana_charset.h
Fix conversion from string literal to 'char *' for Unicode
2016-05-31 11:26:01 +02:00
Images.cpp
Some log changes to pass in specific log levels
2016-08-05 10:05:56 +02:00
Images.h
Moved format #define out from HttpRequest.h so more code doesn't need to know the innaers of HttpRequest
2016-08-31 19:14:16 +02:00
IndexList.h
Remove extra semicolons
2016-05-19 18:37:26 +02:00
init.gb.conf
minor make install changes
2014-05-22 18:46:38 -07:00
injectme3
…
injectmedemo
fix sections.cpp to not set root title section
2014-12-11 19:54:33 -08:00
ip.cpp
Remove unused methods
2016-03-02 23:09:50 +01:00
ip.h
Standardize header guards
2016-03-08 22:16:02 +01:00
IPAddressChecks.cpp
Make is_same_network_linkwise() always return false
2016-06-09 12:33:36 +02:00
IPAddressChecks.h
Make is_same_network_linkwise() always return false
2016-06-09 12:33:36 +02:00
JobScheduler.cpp
Handle shutdown better under load
2016-08-23 15:13:45 +02:00
JobScheduler.h
Revert "Copied over changes from broken 'nomerge' branch"
2016-08-01 14:52:50 +02:00
jpegtopnm
…
Json.cpp
Removed gbstrlen()
2016-07-28 17:04:35 +02:00
Json.h
Remove extra semicolons
2016-05-19 18:37:26 +02:00
Lang.cpp
constness
2016-05-22 21:42:38 +02:00
Lang.h
constness
2016-05-22 21:42:38 +02:00
LanguageIdentifier.cpp
Move LanguageIdentifier::guessCountryTLD to a static method. Remove constructor & destructor & global variable (g_langid).
2015-12-22 10:23:19 +01:00
LanguageIdentifier.h
Standardize header guards
2016-03-08 22:16:02 +01:00
libiconv64.a
added 64 bit libiconv64.a
2014-11-14 17:34:11 -08:00
libiconv.a
…
libiconv.la
…
libjpeg.so.62
thumbnail generation support back in.
2014-04-24 10:13:45 -07:00
libnetpbm.so.10
thumbnail generation support back in.
2014-04-24 10:13:45 -07:00
libpng12.so.0
thumbnail generation support back in.
2014-04-24 10:13:45 -07:00
libtiff.so.4
thumbnail generation support back in.
2014-04-24 10:13:45 -07:00
LICENSE
license fix
2014-06-16 13:52:51 -07:00
Linkdb.cpp
Remove always true dedup from Rdb::init
2016-08-31 11:13:31 +02:00
Linkdb.h
Remove IndexList.cpp and unnecessary IndexList.h includes
2016-07-12 16:51:20 +02:00
linkspam.cpp
constness in linkspam
2016-05-30 12:42:01 +02:00
linkspam.h
constness in linkspam
2016-05-30 12:42:01 +02:00
Log.cpp
Added GbMutex class
2016-08-15 16:24:56 +02:00
Log.h
Fix logError
2016-08-31 15:56:02 +02:00
Loop.cpp
Removed commented-out references to g_udpServer2
2016-08-30 14:28:33 +02:00
Loop.h
Removed unused parameter 'silten' from Loop::unregister*Callback()
2016-08-28 13:43:42 +02:00
main.cpp
Get multi layer tagrec (site -> host -> domain).
2016-09-01 13:43:07 +02:00
Makefile
Add configurable xpath for summary
2016-09-01 13:43:07 +02:00
matches2.cpp
#include cleanup of matches2.*
2016-08-10 12:54:45 +02:00
matches2.h
#include cleanup of matches2.*
2016-08-10 12:54:45 +02:00
Matches.cpp
Code style changes
2016-06-29 11:28:58 +02:00
Matches.h
Remove unnecessary forward decl.
2016-03-08 23:25:53 +01:00
max_hosts.h
#include cleanup in Proxy.*
2016-08-10 00:05:30 +02:00
max_niceness.h
Moved MAX_NICENESS from Threads.h to a separate max_niceness.h file
2016-04-28 14:28:14 +02:00
max_words.h
#include cleanup in Phrases.*
2016-08-05 14:31:28 +02:00
Mem.cpp
Fix warning: comparison of unsigned expression < 0 is always false
2016-08-31 11:13:31 +02:00
Mem.h
Modify Mem.cpp size from int32_t to size_t
2016-08-31 11:13:31 +02:00
Msg0.cpp
Use copy of startkey in msg0 instead of poking directly into msg5
2016-08-30 14:32:57 +02:00
Msg0.h
Use rdbid_t in Msg0.*
2016-08-26 10:35:50 +02:00
Msg1.cpp
Encapsulate Rdb
2016-08-31 11:13:31 +02:00
Msg1.h
include header changes
2016-08-26 14:21:47 +02:00
Msg1f.cpp
Removed some asses from UdpServer
2016-08-11 17:22:06 +02:00
Msg1f.h
Fix includes for Msg1f
2016-07-27 01:15:57 +02:00
Msg2.cpp
Make msg::m_whitelists a dynamically-size array
2016-08-22 10:53:52 +02:00
Msg2.h
Make msg::m_whitelists a dynamically-size array
2016-08-22 10:53:52 +02:00
Msg3.cpp
Encapsulate RdbMerge
2016-08-31 11:13:31 +02:00
Msg3.h
Moved Msg3::m_lists[] into Msg3::m_scan[]
2016-08-29 14:23:19 +02:00
Msg3a.cpp
Code style changes
2016-08-11 12:58:51 +02:00
Msg3a.h
Support limiting results to one shard/host
2016-05-13 15:35:00 +02:00
Msg4.cpp
#include cleanup in Repair.*
2016-08-31 19:22:52 +02:00
Msg4.h
Remove flushMsg4Buffers as it was never called
2016-08-08 14:23:37 +02:00
Msg5.cpp
Remove unused niceness args
2016-09-01 12:11:30 +02:00
Msg5.h
Made Msg5::m_startKey/m_endKey private
2016-08-30 14:33:42 +02:00
Msg13.cpp
Removed some asses from UdpServer
2016-08-11 17:22:06 +02:00
Msg13.h
Remove more qa specific code
2016-07-26 15:19:01 +02:00
Msg20.cpp
Removed some asses from UdpServer
2016-08-11 17:22:06 +02:00
Msg20.h
Made ...::registerHandler() more uniform
2016-08-02 13:29:08 +02:00
Msg22.cpp
Removed some asses from UdpServer
2016-08-11 17:22:06 +02:00
Msg22.h
Removed gbstrlen()
2016-07-28 17:04:35 +02:00
Msg39.cpp
steps to make intersection code clearer
2016-08-30 15:18:21 +02:00
Msg39.h
Moved read-size logic from issuing instance / Msg39Request ot the processing instance
2016-08-01 16:14:34 +02:00
Msg40.cpp
constness on local static
2016-08-19 14:01:26 +02:00
Msg40.h
Remove commented out code / unused include
2016-05-10 16:15:34 +02:00
Msg51.cpp
Make Msg51::slot dynamically sized array
2016-08-19 17:13:47 +02:00
Msg51.h
Make Msg51::slot dynamically sized array
2016-08-19 17:13:47 +02:00
MsgC.cpp
Removed commented-out references to g_udpServer2
2016-08-30 14:28:33 +02:00
MsgC.h
Removed unused members of MsgC (legacy for proxy forwarding?)
2016-08-09 12:48:51 +02:00
Msge0.cpp
include header changes
2016-08-26 14:21:47 +02:00
Msge0.h
include header changes
2016-08-26 14:21:47 +02:00
Msge1.cpp
Fix warning from cppcheck -> The function 'doneAddingTagWrapper' is never used.
2016-08-08 14:23:37 +02:00
Msge1.h
Remove more qa specific code
2016-07-26 15:19:01 +02:00
MsgType.h
Remove dependency of UdpSlots from PageSockets
2016-08-11 12:58:51 +02:00
Multicast.cpp
Instead of using extraInfo which needs to be filled in for every slot, peek into msg instead when PageSocket is called
2016-08-11 12:58:52 +02:00
Multicast.h
Instead of using extraInfo which needs to be filled in for every slot, peek into msg instead when PageSocket is called
2016-08-11 12:58:52 +02:00
mysynonyms.txt
mysyn fixes
2015-04-22 08:34:29 -06:00
PageAddColl.cpp
Removed gbstrlen()
2016-07-28 17:04:35 +02:00
PageAddUrl.cpp
Simplify Parms.h
2016-08-05 14:42:28 +02:00
PageBasic.cpp
localtime() -> localtime_r()
2016-08-09 16:24:54 +02:00
PageCrawlBot.cpp
Remove unused variable
2016-07-22 14:25:31 +02:00
PageCrawlBot.h
Remove diffbot specific CollectionRec::m_notifyUrl
2016-07-22 13:59:55 +02:00
PageGet.cpp
More thread safety: gmtime() -> gmtime_r(), asctime() -> asctime_r()
2016-08-09 16:44:39 +02:00
PageHealthCheck.cpp
Replace INT32/INT64 and likes with PRId32 and likes. Add space before definition.
2016-05-20 09:18:32 +02:00
PageHosts.cpp
#include cleanup of TcpServer.h
2016-08-10 18:31:52 +02:00
PageInject.cpp
Cleanup headers for Dir.h. Remove duplicated function to return dir filename
2016-08-31 11:13:31 +02:00
PageInject.h
Remove commented out code, unused functions
2016-08-05 13:53:55 +02:00
PageLogView.cpp
Turn while(1) loops into more conventional for(;;)
2016-08-21 17:35:23 +02:00
PageParser.cpp
#include cleanup of PageParser.*
2016-08-10 18:11:51 +02:00
PageParser.h
#include cleanup of PageParser.*
2016-08-10 18:11:51 +02:00
PagePerf.cpp
#include cleanup of TcpServer.h
2016-08-10 18:31:52 +02:00
PageReindex.cpp
Simplify Parms.h
2016-08-05 14:42:28 +02:00
PageReindex.h
Remove commented out code in PageReindex
2016-05-10 16:15:32 +02:00
PageResults.cpp
include header changes
2016-08-26 14:21:47 +02:00
PageResults.h
Removed unused members from PageResults.h: State0
2016-08-09 12:40:03 +02:00
PageRoot.cpp
Removed gbstrlen()
2016-07-28 17:04:35 +02:00
PageRoot.h
constness in expandHtml()
2016-03-12 16:08:02 +01:00
Pages.cpp
#include cleanup of PingServer.h (XmlDoc.h was including it for no reason)
2016-08-10 13:07:40 +02:00
Pages.h
Remove more qa specific code
2016-07-26 15:19:01 +02:00
PageSockets.cpp
Add constness
2016-08-11 12:58:52 +02:00
PageStats.cpp
Encapsulate Rdb
2016-08-31 11:13:31 +02:00
PageStatsdb.cpp
Thread safety: ctime() -> ctime_r()
2016-08-09 17:04:57 +02:00
PageThreads.cpp
Remove all trace of twin-sync
2016-07-29 13:53:56 +02:00
PageTitledb.cpp
Removed gbstrlen()
2016-07-28 17:04:35 +02:00
Parms.cpp
Remove unused niceness arg
2016-08-31 11:13:31 +02:00
Parms.h
Simplify Parms.h
2016-08-05 14:42:28 +02:00
Phrases.cpp
#include cleanup of Words.h
2016-08-11 16:51:34 +02:00
Phrases.h
#include cleanup in Phrases.*
2016-08-05 14:31:28 +02:00
PingServer.cpp
Removed some asses from UdpServer
2016-08-11 17:22:06 +02:00
PingServer.h
Removed unused decl for global sendEmail()
2016-08-16 00:25:31 +02:00
pngtopnm
…
pnmscale
…
Pops.cpp
Fix #include in Pops.cpp
2016-08-05 14:31:28 +02:00
Pops.h
Constness in Pops
2016-05-24 12:16:40 +02:00
Pos.cpp
Fix all caps PDF title
2016-06-09 14:52:18 +02:00
Pos.h
constness in Pos
2016-05-24 16:55:59 +02:00
Posdb.cpp
Encapsulate Rdb
2016-08-31 11:13:31 +02:00
Posdb.h
Removed unused Posdb::getNextDocIdSublist()
2016-08-16 00:45:14 +02:00
PosdbTable.cpp
steps to make intersection code clearer
2016-08-30 15:18:21 +02:00
PosdbTable.h
steps to make intersection code clearer
2016-08-30 15:18:21 +02:00
ppmtojpeg
…
Process.cpp
Encapsulate Rdb
2016-08-31 11:13:31 +02:00
Process.h
Add more code to cater for saving rdb index files
2016-08-31 11:13:31 +02:00
Profiler.cpp
#include cleanup in Profiler.*
2016-08-31 19:07:23 +02:00
Profiler.h
#include cleanup in Profiler.*
2016-08-31 19:07:23 +02:00
Proxy.cpp
Remove dependency of UdpSlots from PageSockets
2016-08-11 12:58:51 +02:00
Proxy.h
#include cleanup in Proxy.*
2016-08-10 00:05:30 +02:00
pstotext
…
Punycode.cpp
Start to detect non-asci urls and encode them to ascii.
2015-09-12 15:47:33 -06:00
Punycode.h
Standardize header guards
2016-03-08 22:16:02 +01:00
Query.cpp
Log function will now return void instead of a boolean
2016-08-01 18:12:10 +02:00
Query.h
Fix conversion from string literal to 'char *' for Query
2016-05-31 11:26:01 +02:00
Rdb.cpp
#include cleanup in Repair.*
2016-08-31 19:22:52 +02:00
Rdb.h
Remove unused args/function for Rdb::addRecord
2016-08-31 11:13:31 +02:00
RdbBase.cpp
Add some simple validation on RdbIndex generated file
2016-08-31 15:36:50 +02:00
RdbBase.h
Add locks for RdbBase::m_docIdFileIndex
2016-08-31 11:13:31 +02:00
RdbBuckets.cpp
Remove always true/false args from RdbTree::set and RdbBucket::set (ignoring the only difference that is in test code)
2016-08-31 11:13:31 +02:00
RdbBuckets.h
Remove always true/false args from RdbTree::set and RdbBucket::set (ignoring the only difference that is in test code)
2016-08-31 11:13:31 +02:00
RdbCache.cpp
Cleanup pagestats.cpp mdoification of rdbcache dbname
2016-08-19 14:58:07 +02:00
RdbCache.h
Cleanup pagestats.cpp mdoification of rdbcache dbname
2016-08-19 14:58:07 +02:00
RdbDump.cpp
Generate index file while merging and dumping tree
2016-08-31 11:13:31 +02:00
RdbDump.h
Pass in tree index to RdbDump
2016-08-31 11:13:31 +02:00
rdbid_t.h
Changed char to rdbit_t more places (makes debugging easier)
2016-08-24 14:46:04 +02:00
RdbIndex.cpp
Fix logError
2016-08-31 15:56:02 +02:00
RdbIndex.h
Add some simple validation on RdbIndex generated file
2016-08-31 15:36:50 +02:00
RdbIndexQuery.cpp
Add logic to support time based & size merging of pendingDocIds
2016-08-31 11:13:31 +02:00
RdbIndexQuery.h
Use RdbQueryIndex and removed functions replaced by RdbQueryIndex
2016-08-31 11:13:31 +02:00
RdbList.cpp
Remove unused niceness args
2016-09-01 12:11:30 +02:00
RdbList.h
Remove unused niceness args
2016-09-01 12:11:30 +02:00
RdbMap.cpp
Add some simple validation on RdbIndex generated file
2016-08-31 15:36:50 +02:00
RdbMap.h
Remove now unused RdbMap::m_generatingMap variable
2016-08-31 11:13:31 +02:00
RdbMem.cpp
Encapsulate Rdb
2016-08-31 11:13:31 +02:00
RdbMem.h
Fix indentation, bracing, out-commented code, etc in RdbMem.cpp
2016-08-05 13:47:50 +02:00
RdbMerge.cpp
Pass in tree index to RdbDump
2016-08-31 11:13:31 +02:00
RdbMerge.h
Remove extra semicolon
2016-08-31 11:13:31 +02:00
RdbScan.cpp
Use rdbid_t in RdbScan.*
2016-08-25 17:33:56 +02:00
RdbScan.h
Use rdbid_t in RdbScan.*
2016-08-25 17:33:56 +02:00
RdbTree.cpp
Add some simple validation on RdbIndex generated file
2016-08-31 15:36:50 +02:00
RdbTree.h
Add some simple validation on RdbIndex generated file
2016-08-31 15:36:50 +02:00
README.md
update README.md
2015-03-19 23:31:09 -06:00
Rebalance.cpp
Encapsulate Rdb
2016-08-31 11:13:31 +02:00
Rebalance.h
Rebalance::getNeedsRebalance() should return const
2016-05-22 22:22:56 +02:00
Repair.cpp
#include cleanup in Repair.*
2016-08-31 19:22:52 +02:00
Repair.h
#include cleanup in Repair.*
2016-08-31 19:22:52 +02:00
RobotRule.cpp
Fix cppcheck warning -> Array index 'needlePos' is used before limits check
2016-08-08 14:23:37 +02:00
RobotRule.h
Normalize url. Don't encode character which are not suppose to be encoded.
2016-05-10 16:15:31 +02:00
Robots.cpp
(flexelint) change 'static const char *' in to 'static cons tchar []'
2016-08-21 17:22:13 +02:00
Robots.h
Change rules from using std::list to std::vector
2016-04-05 23:21:50 +02:00
S99gb
added S99gb for loading at boot.
2014-06-23 07:32:38 -06:00
SafeBuf.cpp
Use rdbid_t in Safebuf
2016-08-25 00:24:07 +02:00
SafeBuf.h
Use rdbid_t in Safebuf
2016-08-25 00:24:07 +02:00
Sanity.cpp
Move g_errno_init into g_errno_location itself
2016-07-28 11:23:30 +02:00
Sanity.h
flexelint support in #else
2016-08-28 13:25:45 +02:00
ScalingFunctions.cpp
Move ScalingFunctions unit test to test/unit
2016-03-18 02:45:44 +01:00
ScalingFunctions.h
Standardize header guards
2016-03-08 22:16:02 +01:00
ScopedLock.h
Added GbMutex class
2016-08-15 16:24:56 +02:00
SearchInput.cpp
Log function will now return void instead of a boolean
2016-08-01 18:12:10 +02:00
SearchInput.h
Support fx_qid. Carried all the way through to msg39request
2016-06-17 14:03:31 +02:00
Sections.cpp
Fix warnings
2016-07-26 22:59:59 +02:00
Sections.h
Remove more qa specific code
2016-07-26 15:19:01 +02:00
SiteGetter.cpp
Removed gbstrlen()
2016-07-28 17:04:35 +02:00
SiteGetter.h
Remove extra semicolons
2016-05-19 18:37:26 +02:00
sitelinks.txt
fixed missing sites in sitelinks.txt
2015-03-05 20:32:01 -08:00
sort.cpp
Removed quickpoll from sort.cpp (and inclusion of Loop.h)
2016-08-10 12:50:34 +02:00
sort.h
use gbmemcpy not memcpy so we can get profiler working again
2015-01-13 12:25:42 -07:00
Speller.cpp
Log function will now return void instead of a boolean
2016-08-01 18:12:10 +02:00
Speller.h
Speller::getPhrasePopularity() const
2016-05-23 15:21:15 +02:00
Spider.cpp
Encapsulate RdbTree
2016-08-31 11:13:31 +02:00
Spider.h
constness; and removed unused functions in Spider.*
2016-08-25 16:50:18 +02:00
SpiderColl.cpp
Remove always true/false args from RdbTree::set and RdbBucket::set (ignoring the only difference that is in test code)
2016-08-31 11:13:31 +02:00
SpiderColl.h
Remove more qa specific code
2016-07-26 15:19:01 +02:00
SpiderLoop.cpp
Encapsulate Rdb
2016-08-31 11:13:31 +02:00
SpiderLoop.h
Remove Msg12
2016-06-29 12:34:23 +02:00
SpiderProxy.cpp
Removed some asses from UdpServer
2016-08-11 17:22:06 +02:00
SpiderProxy.h
Remove diffbot specific code. -> maxcrawl, maxprocess, crawl pattern, crawl done notification, etc...
2016-07-22 13:59:55 +02:00
Statistics.cpp
Revert "Use std::map::at instead of []"
2016-08-31 11:13:31 +02:00
Statistics.h
Rename Statistics header guard to prevent warning from clang
2016-07-22 13:59:55 +02:00
Stats.cpp
Remove g_stats.addSpiderPoint and related spider statistics
2016-07-25 17:03:51 +02:00
Stats.h
Removal/comment of unused members
2016-08-09 20:59:54 +02:00
Statsdb.cpp
Remove unused args/function for Rdb::addRecord
2016-08-31 11:13:31 +02:00
Statsdb.h
#include cleanup of Statsdb.*
2016-08-10 13:17:00 +02:00
StopWords.cpp
Make isCommonWord() thread-safe.
2016-08-19 14:21:07 +02:00
StopWords.h
#include cleanup of StopWords.h
2016-08-11 16:43:25 +02:00
Summary.cpp
Add configurable xpath for summary
2016-09-01 13:43:07 +02:00
Summary.h
Add configurable xpath for summary
2016-09-01 13:43:07 +02:00
SummaryCache.cpp
Optimize summary cache when it is effectively disabled
2016-03-22 12:23:10 +01:00
SummaryCache.h
Standardize header guards
2016-03-08 22:16:02 +01:00
Synonyms.cpp
First batch of changes streamlining emergency shutdown code
2016-06-20 12:30:26 +02:00
Synonyms.h
Synonyms::addStripped() const
2016-05-23 15:55:00 +02:00
Tagdb.cpp
Add configurable xpath for summary
2016-09-01 13:43:07 +02:00
Tagdb.h
Get multi layer tagrec (site -> host -> domain).
2016-09-01 13:43:07 +02:00
TcpServer.cpp
#include cleanup in Profiler.*
2016-08-31 19:07:23 +02:00
TcpServer.h
#include cleanup of TcpServer.h
2016-08-10 18:31:52 +02:00
TcpSocket.h
Removed unused READ_CHUNK_SIZE
2016-08-16 00:40:07 +02:00
tifftopnm
…
Timezone.cpp
Make timezone initialization thread-safe
2016-08-19 16:02:03 +02:00
Timezone.h
#include cleanup of TimeZone.*
2016-08-19 16:00:36 +02:00
Title.cpp
Removed gbstrlen()
2016-07-28 17:04:35 +02:00
Title.h
Standardize header guards
2016-03-08 22:16:02 +01:00
Titledb.cpp
Remove always true dedup from Rdb::init
2016-08-31 11:13:31 +02:00
Titledb.h
Encapsulate Rdb
2016-08-31 11:13:31 +02:00
TitleRecVersion.h
Add a more restrictive check for who query parameter
2016-05-10 16:15:31 +02:00
TopTree.cpp
Remove always true/false args from RdbTree::set and RdbBucket::set (ignoring the only difference that is in test code)
2016-08-31 11:13:31 +02:00
TopTree.h
#include cleanup
2016-07-11 15:13:59 +02:00
types.h
Renamed KEYADD to KEYINC and KEYSUB to KEYDEC
2016-08-04 13:49:40 +02:00
UCPropTable.cpp
Log function will now return void instead of a boolean
2016-08-01 18:12:10 +02:00
UCPropTable.h
constness in UCPropTable
2016-05-30 16:47:04 +02:00
UdpProtocol.h
#include cleanup in File.h (and resulting other cleanup)
2016-08-10 00:32:00 +02:00
UdpServer.cpp
Removed write-only variable g_callSlot
2016-08-19 12:14:21 +02:00
UdpServer.h
Removed some asses from UdpServer
2016-08-11 17:22:06 +02:00
UdpSlot.cpp
Add constness
2016-08-11 13:04:53 +02:00
UdpSlot.h
Use m_activeListNext directly
2016-08-11 16:45:17 +02:00
UdpStatistic.cpp
Fix warning: no newline at end of file
2016-08-31 11:13:31 +02:00
UdpStatistic.h
Remove additional semicolon
2016-08-12 10:41:26 +02:00
Unicode.cpp
Log function will now return void instead of a boolean
2016-08-01 18:12:10 +02:00
Unicode.h
Handle unicode codepoints >= U+10000 correctly
2016-06-07 12:19:17 +02:00
UnicodeProperties.cpp
Log function will now return void instead of a boolean
2016-08-01 18:12:10 +02:00
UnicodeProperties.h
Removed unused enum HangulComposition
2016-07-27 01:18:42 +02:00
unifiedDict.txt
…
Url.cpp
Make url extension initialization thread-safe
2016-08-19 16:09:34 +02:00
Url.h
Add constness to some tld/domain/url functions
2016-08-11 12:58:51 +02:00
UrlComponent.cpp
Fix redundant condition: '!A || (A && B)' is equivalent to '!A || B'
2016-06-21 13:15:12 +02:00
UrlComponent.h
Add print functions for ease of debugging
2016-05-27 12:22:37 +02:00
UrlParser.cpp
Fix bug when handling extra long url (url longer than MAX_URL_LEN)
2016-07-05 12:19:33 +02:00
UrlParser.h
Fix bug when handling extra long url (url longer than MAX_URL_LEN)
2016-07-05 12:19:33 +02:00
valgrind.cfg
added more helgrind suppressions (until we have time to look into them)
2016-08-25 15:49:57 +02:00
Version.cpp
Print git branch as well
2016-08-31 11:13:31 +02:00
Version.h
Add more logging on git commit id (so we can match logs with binary)
2016-06-28 12:27:07 +02:00
Wiki.cpp
#include cleanup in File.h (and resulting other cleanup)
2016-08-10 00:32:00 +02:00
Wiki.h
More constness in Wiki
2016-05-24 12:16:40 +02:00
wikititles.txt.part1
…
wikititles.txt.part2
…
wiktionary-buf.txt
when user searches for a word without the
2014-06-01 09:37:00 -07:00
wiktionary-lang.txt
when user searches for a word without the
2014-06-01 09:37:00 -07:00
wiktionary-syns.dat
when user searches for a word without the
2014-06-01 09:37:00 -07:00
Wiktionary.cpp
#include cleanup in File.h (and resulting other cleanup)
2016-08-10 00:32:00 +02:00
Wiktionary.h
Fix unreachable-code warnings
2016-06-28 11:48:30 +02:00
Words.cpp
#include cleanup of Words.h
2016-08-11 16:51:34 +02:00
Words.h
#include cleanup of Words.h
2016-08-11 16:51:34 +02:00
Xml.cpp
Log function will now return void instead of a boolean
2016-08-01 18:12:10 +02:00
Xml.h
Fix unreachable-code warnings
2016-06-28 11:48:30 +02:00
XmlDoc_Indexing.cpp
Removed gbstrlen()
2016-07-28 17:04:35 +02:00
XmlDoc.cpp
Add configurable xpath for summary
2016-09-01 13:43:07 +02:00
XmlDoc.h
#include cleanup of PingServer.h (XmlDoc.h was including it for no reason)
2016-08-10 13:07:40 +02:00
XmlNode.cpp
Removed gbstrlen()
2016-07-28 17:04:35 +02:00
XmlNode.h
More constness in XmlNode
2016-05-24 13:08:55 +02:00
XmlXPath.cpp
Add configurable xpath for summary
2016-09-01 13:43:07 +02:00
XmlXPath.h
Add configurable xpath for summary
2016-09-01 13:43:07 +02:00
zconf.h
updated to a new libz64.a. updated zconf.h and
2014-11-17 14:53:15 -08:00

README.md

open-source-search-engine

An open source web and enterprise search engine and spider/crawler. As can be seen on http://www.gigablast.com/ .

RUNNING GIGABLAST

See html/faq.html for all administrative documentation including the quick start instructions.

Alternatively, visit http://www.gigablast.com/faq.html

CODE ARCHITECTURE

See html/developer.html for all code documentation.

Alternatively, visit http://www.gigablast.com/developer.html

CONTACT

Contact me for feature requests or help in general. I will work for free for good use cases. mattdwells@hotmail.com.

Description
No longer maintained. Please read our shutdown message.
search
Readme 254 MiB
Languages
C++ 87%
JavaScript 6.5%
HTML 3.7%
C 1%
Python 0.6%
Other 1.1%