# 抖音网页爬虫

**Repository Path**: LSE_QX/tiktok-crawler

## Basic Information

- **Project Name**: 抖音网页爬虫
- **Description**: 基于 webclient 与 selenium 实现的抖音网页端爬取
- **Primary Language**: Unknown
- **License**: Apache-2.0
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 12
- **Forks**: 4
- **Created**: 2021-09-26
- **Last Updated**: 2024-01-31

## Categories & Tags

**Categories**: Uncategorized

**Tags**: Java, htmlunit, Selenium, Jsoup

## README

# 抖音网页爬虫

#### 介绍
基于 webclient 与 selenium 实现的抖音网页端爬取

#### 软件架构
webclient + selenium + java + jsoup

#### 备注说明

本仓库仅含有可执行java类文件

##### POM

```
<!-- selenium 模拟浏览器自动化测试 -->
<!-- https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-java -->
<dependency>
    <groupId>org.seleniumhq.selenium</groupId>
    <artifactId>selenium-java</artifactId>
    <version>4.0.0-beta-4</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-chrome-driver -->
<dependency>
    <groupId>org.seleniumhq.selenium</groupId>
    <artifactId>selenium-chrome-driver</artifactId>
    <version>4.0.0-beta-4</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.google.guava/guava -->
<dependency>
    <groupId>com.google.guava</groupId>
    <artifactId>guava</artifactId>
    <version>30.1.1-jre</version>
</dependency>

        <!-- htmlunit 页面分析工具(可模拟浏览器运行) -->
        <!-- https://mvnrepository.com/artifact/net.sourceforge.htmlunit/htmlunit -->
        <dependency>
            <groupId>net.sourceforge.htmlunit</groupId>
            <artifactId>htmlunit</artifactId>
            <version>2.33</version>
        </dependency>
        
        
        <!-- okhttp3 -->
        <!-- https://mvnrepository.com/artifact/com.squareup.okhttp3/okhttp -->
        <dependency>
            <groupId>com.squareup.okhttp3</groupId>
            <artifactId>okhttp</artifactId>
            <version>3.10.0</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/com.squareup.okio/okio -->
        <dependency>
            <groupId>com.squareup.okio</groupId>
            <artifactId>okio</artifactId>
            <version>2.3.0</version>
        </dependency>

        <!-- jsoup html解析器 -->
        <!-- https://mvnrepository.com/artifact/org.jsoup/jsoup -->
        <dependency>
            <groupId>org.jsoup</groupId>
            <artifactId>jsoup</artifactId>
            <version>1.11.3</version>
        </dependency>
```



##### chromedriver 下载地址

[chromedriver.storage.googleapis.com/index.html](http://chromedriver.storage.googleapis.com/index.html)