Data-intensive text processing with MapReduce
by Jimmy Lin · first published 2010
8mentions
5recommended by
6HN commenters
2009–2015first – last mentioned
83% recommend17% neutral
Why HN recommends it
A practical introduction to designing text-processing algorithms for MapReduce, covering integer compression and large-scale indexing. It is useful for engineers moving into distributed data work.
Summarised by Claude from recommending comments
Affiliate links. As an Amazon Associate I earn from qualifying purchases.
Mentions per year
What Hacker News says
Related books
2007–2021Data mining
I. H. Witten, Ian H. Witten, Eibe Frank
Recommended by 9 · mentioned by 149 recommend · 14 mention9 recommend · 149 recommend
2008–2026Managing gigabytes
Ian H. Witten, Alistair Moffat, Timothy C. Bell
Recommended by 26 · mentioned by 3526 recommend · 35 mention26 recommend · 3526 recommend
…add a note that execution of queries with Cartesian product may be extremely slow (with Map Reduce or other parallelization technology. Let the database vendors compete in that field). I recommend book “Data-Intensive Text Processing with MapReduce” by Jimmy Lin and Chris Dyer book for further reading on how to implement relational joins with MapReduce. DML and DDL will be changed substantially, though. Drop or relax relational constraints, different…
on “CoSQL: SQL and noSQL are really just two sides of the same coin” ↗I have found the best resource for map reduce algorithms to be "Data-Intensive Text Processing with MapReduce" by Chris Dyer and Jimmy Lin - a short and very useful book that characterizes map reduce problems and their solutions.
on “Google's Introduction to Parallel Programming and MapReduce” ↗…(Bell) Natural Language Processing (Manning) Natural Language Understanding (Allen) Speech and Language Processing (Jurafsky) The Text Mining Handbook (Sanger) Statistical Machine Translation (Koehn) Data-Intensive Text Processing with MapReduce (Lin) Algorithms on strings (Gusfield) Jewels of Stringology (Crochemore) Regular Expressions (Friedl), also: http://swtch.com/~rsc/regexp/regexp1.html and automata theory (Hopcroft) Practical Text…
on “Ask HN: Best Text Mining Resources” ↗Data Intensive Text Processing with MapReduce. It was a key reason I got my current job, which is my dream job at this point in my career.
on “Ask HN: What book changed your life in 2013?” ↗…(http://ocw.mit.edu/courses/mathematics/18-06sc-linear-algebr...) * Natural Language Processing (Coursera has several courses) * Machine Learning/Information Retrieval/Data Mining (I used a couple books, Mining Massive Datasets and Data-Intensive Text Processing with MapReduce being the two I'd recommend) * Networking (Coursera has a good course) * Operating Systems (https://www.youtube.com/watch?v=XgQo4JkN4Bw&list=PL62A66DDD3...) * Distributed Systems (I read Tanenbaum's…
on “List of University Courses for Learning Computer Science” ↗…recommendations. I have read about 20 books on his lists and have them in my library. The difference for me is that some of these were very difficult for me to work through! Off topic, but: I am reading "Data-Intensive Text Processing with MapReduce" right now and can strongly recommend it: it characterizes different types of problems that can be solved with map reduce, possible solutions, and tradeoffs for different approaches. Really a useful little…
on “Peter Norvig's Library” ↗Nice summary. Slightly off topic, I first learned about the basics of integer compression with some good context in "Data Intensive Text Processing with MapReduce", which I recommend not necessarily for that section, but otherwise as a great book. http://lintool.github.com/MapReduceAlgorithms/
on “Fast integer compression: decoding billions of integers per second” ↗And you should avoid "tiny" and other homebrewed Map/Reduce implementations for any serious work. This advice according to Prof. Jimmy Lin during his "Data-Intensive Text Processing with MapReduce" tutorial that I took at the NAACL/HLT natural language processing conference: http://www.umiacs.umd.edu/~jimmylin/cloud-computing/NAACL-HL...
on “Really Understanding the MapReduce Model” ↗