R 网页数据爬虫1

1.WHY R？

#1.FOR a software environment with a primarily statistical focus.

#2.there will be an amazing visual work.

#May be a complete set of operational procedures.

2.About basics.

we need threw ourselves into the preparation with some basic knowledge of HTML, XML and the logic of regular expressions and Xpath, BUT the operations are executed from WIHTIN R!

3.RECOMMENDATION

http://www.r-datacollection.com

4.A little case study.

#爬取电影票房信息

library(stringr)

library(XML)

library(maps)

#htmlParse()用来interpreting HTML

#创建一个object

movie_parsed<-htmlParse("http://58921.com/boxoffice/wangpiao/20161004",

                        encoding = "UTF-8")

#the next step:extract tables/data

#readHTMLTable() for identifying and reading out those tables

tables<-readHTMLTable(movie_parsed,stringsAsFactors=FALSE)

is.matrix(tables)

is.character(tables)

is.data.frame(tables)

is.list(tables)

#so we got an "list" format#

因为R对于中文的支持不是很好，所以碰到一些中文乱码是正常的，所以我们需要more advanced text manipulation tools.（本例中出现了部分列信息的完全丢失是因为该网站的某些列的数据是以.png格式放置的。）

5.ABC's of...

For browsing the Web, there is a hidden standard behind the scenes that structures how information is displayed.

#HTML or the hypertext markup language

Not a dedicated data storage format, but usually contains the useful information. And in general HTML is used to shape the display of information.

#XML the extensible markup language or XML

The main purpose of XML is to storage data. Thus HTML documents are interpreted and transformed in to pretty-looking output by browsers, whereas XML is "just" data wrapped in user-defined tags. The user-defined tags make XML much more flexible for storing data than HTML. Both HTML and XML-style document offer natrual, often hierarchical, structures for data storage.

(unfinished......)

#JSON or JavaScript Object Notation

基于JavaScript语言的轻量级的数据交换格式

#AJAX or "Asynchronous JavaScript and XML"

____________________________________________________________________________________________

HTTP	R
XML/HTML	XPath
JSON	JSON parsers
AJAX	Selenuim
Plain text	Regular expressions

巴特西

R 网页数据爬虫1

最新文章

热门文章