支持 #5342: Architecture and Design Document 支持 #5341: User Manual and Test Document 缺陷 #5145: unack message 缺陷 #5270: the var of lastdomain always is null 缺陷 #5129: deliver to priority queue 缺陷 #5144: listHtml.getHtml() may be null 缺陷 #5143: code error 403 (403 Forbidden) 缺陷 #5142: Downloader may shutdown when the download method throw a Exception 缺陷 #5141: task message shouldn't ack 2 times 缺陷 #5057: run it on the server 缺陷 #5055: the parameter of domain transmit error 缺陷 #5041: DAO resource can't init 缺陷 #5040: share the variable of lastDomain、lastTime result in rate control failure 缺陷 #5039: separate download thread 缺陷 #5037: channel error 缺陷 #5036: 'read timed out' cause thread stop 缺陷 #4991: Table 'html_crawler_test.*' doesn't exist 缺陷 #4989: channel.basicPublish error on retry process 缺陷 #4990: java.lang.NullPointerException (out of scope) 缺陷 #4988: AlreadyClosedException: channel is already closed due to channel error 功能 #4935: downloader get queue name from site_queue 功能 #4987: automatic recovery mechanisms on downloader 功能 #4986: use thread heart Beat to monitor downloader 功能 #4985: thread-safe resource on downloader 功能 #4959: flow chart of Extractor 功能 #4723: enhance the extractor module 任务 #4917: test rate control 功能 #4725: failure to retry 功能 #4804: use json message encapsulation 功能 #4802: add attr of errorTimes to TaskMessage 任务 #4727: improved task generator v2 功能 #4722: enhance the capacity of task generator module 功能 #4724: consider the multiple queue 功能 #4720: use thread pool 功能 #4770: enhance downloader 任务 #4764: add SQL script (needed) into git 周报 #4631: adjust architecture 功能 #4716: Do we need to design a monitor for distributed nodes? 任务 #4721: monitor the status of downloader 任务 #4699: what to optimize next? 功能 #4667: Serialization of message 功能 #4661: EXCHANGE mechanism 功能 #4663: build reasonable strategy about task deliver to multi queue 缺陷 #4660: about prefetch count 任务 #4659: test sending and receiving task 缺陷 #4646: the downloader will be blocked if an exception occurs 缺陷 #4645: encapsulate task message parsing process 功能 #4644: rate control 功能 #4630: Is the extract component still need mongodb API ? 任务 #4530: development tools 任务 #4550: xml file NullPointerException 任务 #4555: TableHelper's function 任务 #4568: new function of TableHelper:creating tables automatically 功能 #4569: a simple flow chat of task generator module 任务 #4570: Sending generated task to MQ 任务 #4571: what tables should MySql have? 缺陷 #4580: ensure that every task can be consumed by downloader 功能 #4579: improve the task message encapsulation 功能 #4572: the download page deliver to corresponding table automatically 缺陷 #4567: org.springframework.jdbc.BadSqlGrammarException in TaskGenerator class 功能 #4566: build extract components 任务 #4565: test the prototype of sending and receiving 缺陷 #4564: encapsulate the task message 功能 #4562: the mode of downloader change from pull to push 功能 #4560: the function of QueueingConsumer 支持 #4557: Configuration ans SubSite 任务 #4551: access to db 缺陷 #4556: after each mode is over 功能 #4552: is multi-mode necessary? 功能 #4554: message protocol 缺陷 #4553: java 535 Error: authentication failed 功能 #4542: parsing xml 支持 #4549: choice of extractor's location 缺陷 #4548: model of site to be crawled 功能 #4546: build task queue model 功能 #4531: task generator 功能 #4541: research MQ , build the MQ module 缺陷 #4539: Error: Java: not the end of the string literal 缺陷 #4536: can't reslove method 'getBatch(String,int,int)' in class 'DetailUrlDispather' 任务 #4361: New architecture design 功能 #4527: Design and implement the task generator 任务 #4489: design db 任务 #4379: compare and chose a message queue 周报 #4474: Progress Report(27 July)
|
« 上一页 | 下一页 » |