Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[add][plugin] Add support for SAP HANA database #977

Merged
merged 1 commit into from
Dec 6, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions docs/assets/jobs/hanareader.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
{
"job": {
"content": [
{
"reader": {
"name": "hanareader",
"parameter": {
"column": [
"*"
],
"connection": [
{
"jdbcUrl": [
"jdbc:sap://wgzhao-pc:39017/system"
],
"table": [
"addax_tbl"
]
}
],
"username": "system",
"password": "HXEHana1"
}
},
"writer": {
"name": "streamwriter",
"parameter": {
"print": true
}
}
}
],
"setting": {
"speed": {
"bytes": -1,
"channel": 1
}
}
}
}
58 changes: 58 additions & 0 deletions docs/assets/jobs/hanawriter.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
{
"job": {
"setting": {
"speed": {
"channel": 1,
"bytes": -1
}
},
"content": {
"reader": {
"name": "streamreader",
"parameter": {
"column": [
{
"value": "Addax",
"type": "string"
},
{
"value": 19880808,
"type": "long"
},
{
"value": "1988-08-08 08:08:08",
"type": "date"
},
{
"value": true,
"type": "bool"
},
{
"value": "test",
"type": "bytes"
}
],
"sliceRecordCount": 1000
}
},
"writer": {
"name": "hanawriter",
"parameter": {
"connection": [
{
"jdbcUrl": "jdbc:sap://wgzhao-pc:39017/system",
"table": [
"addax_tbl"
]
}
],
"username": "system",
"password": "HXEHana1",
"column": [
"*"
]
}
}
}
}
}
28 changes: 28 additions & 0 deletions docs/reader/hanareader.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# HANA Reader

HANAReader 插件实现了从 SAP HANA 读取数据的能力

## 示例


下面的配置是读取该表到终端的作业:

=== "job/hanareader.json"

```json
--8<-- "jobs/hanareader.json"
```

将上述配置文件保存为 `job/hana2stream.json`

### 执行采集命令

执行以下命令进行数据采集

```shell
bin/addax.sh job/hana2stream.json
```

## 参数说明

HANAReader 基于 [rdbmsreader](../rdbmsreader) 实现,因此可以参考 rdbmsreader 的所有配置项。
40 changes: 40 additions & 0 deletions docs/writer/hanawriter.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# HANA Writer

HANAWriter 插件实现了写入数据到 [SAP HANA][1] 目的表的功能。

## 示例

假定要写入的 HANA 表建表语句如下:

```sql
create table system.addax_tbl
(
col1 varchar(200) ,
col2 int(4),
col3 date,
col4 boolean,
col5 clob
);
```

这里使用一份从内存产生到 HANA 导入的数据。

=== "job/hanawriter.json"

```json
--8<-- "jobs/hanawriter.json"
```

将上述配置文件保存为 `job/hana2stream.json`

### 执行采集命令

执行以下命令进行数据采集

```shell
bin/addax.sh job/hana2stream.json
```

## 参数说明

HANAWriter 基于 [rdbmswriter](../rdbmswriter) 实现,因此可以参考 rdbmswriter 的所有配置项。
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,8 @@ public enum DataBaseType
Trino("trino", "io.trino.jdbc.TrinoDriver"),
Sybase("sybase", "com.sybase.jdbc4.jdbc.SybDriver"),
Databend("databend", "com.databend.jdbc.DatabendDriver"),
Access("access","net.ucanaccess.jdbc.UcanaccessDriver");
Access("access","net.ucanaccess.jdbc.UcanaccessDriver"),
HANA("hana", "com.sap.db.jdbc.Driver");

private static final Pattern jdbcUrlPattern = Pattern.compile("jdbc:\\w+:(?:thin:url=|//|thin:@|)([\\w\\d.,]+).*");

Expand Down
16 changes: 16 additions & 0 deletions package.xml
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,14 @@
<fileMode>0644</fileMode>
<outputDirectory>addax-${project.version}</outputDirectory>
</fileSet>
<fileSet>
<directory>plugin/reader/hanareader/target/hanareader-${project.version}/</directory>
<includes>
<include>**/*.*</include>
</includes>
<fileMode>0644</fileMode>
<outputDirectory>addax-${project.version}</outputDirectory>
</fileSet>
<fileSet>
<directory>plugin/reader/hbase11xreader/target/hbase11xreader-${project.version}/</directory>
<includes>
Expand Down Expand Up @@ -402,6 +410,14 @@
<fileMode>0644</fileMode>
<outputDirectory>addax-${project.version}</outputDirectory>
</fileSet>
<fileSet>
<directory>plugin/writer/hanawriter/target/hanawriter-${project.version}/</directory>
<includes>
<include>**/*.*</include>
</includes>
<fileMode>0644</fileMode>
<outputDirectory>addax-${project.version}</outputDirectory>
</fileSet>
<fileSet>
<directory>plugin/writer/hbase11xsqlwriter/target/hbase11xsqlwriter-${project.version}/</directory>
<includes>
Expand Down
44 changes: 44 additions & 0 deletions plugin/reader/hanareader/package.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
<assembly
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2 http://maven.apache.org/xsd/assembly-component-1.1.2.xsd">
<id>release</id>
<formats>
<format>dir</format>
</formats>
<includeBaseDirectory>false</includeBaseDirectory>
<fileSets>
<fileSet>
<directory>src/main/resources</directory>
<includes>
<include>*.json</include>
</includes>
<outputDirectory>plugin/reader/${project.artifactId}</outputDirectory>
</fileSet>
<fileSet>
<directory>target/</directory>
<includes>
<include>${project.artifactId}-${project.version}.jar</include>
</includes>
<outputDirectory>plugin/reader/${project.artifactId}</outputDirectory>
</fileSet>
<fileSet>
<directory>src/main/libs</directory>
<includes>
<include>*.*</include>
</includes>
<outputDirectory>plugin/reader/${project.artifactId}/libs</outputDirectory>
</fileSet>
</fileSets>

<dependencySets>
<dependencySet>
<useProjectArtifact>false</useProjectArtifact>
<outputDirectory>plugin/reader/${project.artifactId}/libs</outputDirectory>
<scope>runtime</scope>
<excludes>
<exclude>com.wgzhao.addax:*</exclude>
</excludes>
</dependencySet>
</dependencySets>
</assembly>
67 changes: 67 additions & 0 deletions plugin/reader/hanareader/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<artifactId>addax-all</artifactId>
<groupId>com.wgzhao.addax</groupId>
<version>4.1.4-SNAPSHOT</version>
<relativePath>../../../pom.xml</relativePath>
</parent>

<artifactId>hanareader</artifactId>
<name>hanareader-reader</name>
<description>SAP HANA Reader</description>
<packaging>jar</packaging>

<dependencies>

<dependency>
<groupId>com.wgzhao.addax</groupId>
<artifactId>addax-common</artifactId>
<version>${project.version}</version>
<exclusions>
<exclusion>
<artifactId>slf4j-log4j12</artifactId>
<groupId>org.slf4j</groupId>
</exclusion>
</exclusions>
</dependency>

<dependency>
<groupId>com.wgzhao.addax</groupId>
<artifactId>addax-rdbms</artifactId>
<version>${project.version}</version>
</dependency>

<dependency>
<groupId>com.sap.cloud.db.jdbc</groupId>
<artifactId>ngdbc</artifactId>
<version>2.18.16</version>
</dependency>

</dependencies>

<build>
<plugins>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<descriptors>
<descriptor>package.xml</descriptor>
</descriptors>
<finalName>${project.artifactId}-${project.version}</finalName>
</configuration>
<executions>
<execution>
<id>release</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>

</project>
Loading
Loading