2e4d80c80e
Merge branch 'master' into nomerge2
2017-03-14 15:41:32 +01:00
4af955e174
Move document indexing from main thraed to a job
...
XmlDoc::indexDoc/indexDoc2/getMetaList() was being called from a callback in main thraed. Move that work to a thrad. Due to the multiple async calls done by getMetaList() there will be some overhead by this but at least the main thread won't be clogged when encountering 5MB+ documents anymore.
2017-03-14 15:23:15 +01:00
1bc06e97c5
Merge branch 'master' into nomerge2
2017-03-09 17:02:43 +01:00
7592d260c6
Move generateGlobalIndex to thread; Use updateGlobalIndex after merging instead of regenerating global index
2017-03-09 16:59:43 +01:00
8ee096f84c
Merge branch 'master' into nomerge2
2017-03-07 12:11:43 +01:00
7a09cd8870
PageThreads: verify-data job type
2017-03-06 17:24:19 +01:00
d4c150b761
Merge branch 'master' into nomerge2
2017-03-06 16:31:46 +01:00
978988373c
Rename spider-dedup threads to merge-filter. Add filterTitledbList (uncalled) to filter based on urlblocklist
2017-03-06 16:17:49 +01:00
71c8deb6dc
Move dedup spiderdb to cpu thread
2017-03-06 16:11:04 +01:00
7f044bd0e8
Move dedup spiderdb to cpu thread
2017-03-01 14:22:21 +01:00
69d48b7a94
Merge branch 'master' into nomerge2
...
Conflicts:
Parms.cpp
2017-01-29 01:21:42 +01:00
98ed7445c6
Explicit cast to double in division when printing average job times.
2017-01-27 11:27:04 +01:00
d1023debfa
Merge branch 'master' into nomerge2
2017-01-06 15:05:27 +01:00
026f1c1556
Use StackBuf<> instead of direct char[]+SafeBuf
2017-01-06 12:19:21 +01:00
d714e0e0ef
Merge branch 'master' into nomerge2
2016-12-22 15:29:59 +01:00
b3164d629f
Move RdbIndex merge to cpu thread
2016-12-22 15:29:09 +01:00
dab892fc05
Merge branch 'master' into nomerge2
...
Conflicts:
JobScheduler.cpp
JobScheduler.h
Msg2.cpp
Msg39.cpp
Msg40.cpp
UdpServer.cpp
main.cpp
2016-11-15 12:13:43 +01:00
323602f314
Dropped thread_type_unlink enum element
2016-11-10 14:38:24 +01:00
2357623dcb
Merge branch 'master' into nomerge2
2016-09-06 12:54:12 +02:00
e8017aac14
show job count in threads page
2016-09-06 12:53:36 +02:00
70e28231c9
Merge branch 'master' into nomerge2
2016-09-05 17:11:42 +02:00
305e32ddda
put detailed statistics in html comment
2016-09-05 15:08:06 +02:00
9ff99c369f
Merge branch 'master' into nomerge2
...
Conflicts:
RdbList.cpp
RdbList.h
2016-09-05 14:54:49 +02:00
96f2298b55
Maintain statistics for per-job-type times in various phases
2016-09-05 14:50:29 +02:00
fc0debf535
Fix typo on statistics page
2016-09-05 14:50:29 +02:00
495619412b
Show statistics for coordinator jobs/threads
2016-09-05 14:01:49 +02:00
7d5a56e4b8
thread-statistics
2016-08-23 17:16:36 +02:00
766003b6c7
Remove all trace of twin-sync
2016-07-29 13:53:56 +02:00
79ae62881a
Remove all trace of hdtemp
2016-07-29 13:53:56 +02:00
d40ecb2f8e
Replace INT32/INT64 and likes with PRId32 and likes. Add space before definition.
2016-05-20 09:18:32 +02:00
85795f0b45
Make PageThreads show something in html again
2016-05-03 16:33:12 +02:00
be9840d1c0
Replaced Threads.* with a jobscheduler
...
Threads were being created and destroyed which can be expensive. The
thread-per-job model has been changed to a job scheduler that manages the job
queues and threads in pools. The submission of a job now specifies start/finish
routines, state, and as precisely what kind of job it is. The job scheduler then
takes care of the rest. it is hidden how many queues and pools there are.
2016-04-29 14:27:27 +02:00
95f7823149
Removed more commented-out code
2016-04-19 15:40:29 +02:00
97848d4628
constness/getThreadQueue
2016-04-19 15:38:55 +02:00
7c270e6d3b
Removed commented-out code (disk thread statistics)
2016-04-19 15:32:56 +02:00
16d039ac60
More Threads cleanup
2016-04-10 15:36:02 +02:00
5f2b8cb5cd
Make max-threads configurable
2016-04-08 13:33:32 +02:00
c653b0989c
undo some possible averse changes
2015-09-04 11:31:43 -07:00
26404443a8
if disk thread took 0 ms then put *>*XMB/s in thread table
2015-09-03 14:50:56 -06:00
6cf69399e5
handle bad host sending us a c1 request
...
so we dont core
2015-09-03 14:27:33 -06:00
b50f36cd73
a whole new threads stack
2015-09-03 11:59:43 -06:00
4c19453ea9
working with -m32 for basic testing.
...
compiles for 64-bit.
2014-11-12 11:38:37 -08:00
96b8197ad3
now it compiles with -m32
2014-11-10 14:45:11 -08:00
e7dd8f7956
replace long long with int64_t
2014-10-30 13:36:39 -06:00
b755b4d581
formatting fixes
2014-01-19 00:57:20 -08:00
fe3a879758
formatting changes
2014-01-19 00:38:02 -08:00
f6e560c1f4
Initial file population.
2013-08-02 13:12:24 -07:00
