19/05/14scala 大数据作业预备知识

最新推荐文章于 2022-09-08 02:49:55 发布

qq_37175233

最新推荐文章于 2022-09-08 02:49:55 发布

阅读量183

点赞数

CC 4.0 BY-SA版权

本文链接：https://blog.youkuaiyun.com/qq_37175233/article/details/90198911

本文详细介绍了如何在Spring Boot项目中整合Scala语言，包括解决YAML配置问题、搭建Scala开发环境、实现RESTful API、整合数据库以及进行单元测试的过程。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

20190513

scala 大数据作业预备知识

scala 大数据作业预备知识

数据分析大作业20190513期间遇到的问题

i错误点

yml识别问题

file-settings-plugins搜yaml ，下载，重启；
** 还应settings-editor-filetype** 添加*.yml

已经成功了就是那个yml文件图标不是绿的。

Failed to load property source from location ‘classpath:/application.yml’

emmmm这个原因是因为
冒号后面我没加空格

操作过程

基本过程

写好yml
浏览器输入

操作2 hello world

1 在这里插入图片描述
2

整合scala

1 在这里插入图片描述
2
enable auto import
自动导入结2.11.8版本

<plugin>
				<groupId>net.alchim31.maven</groupId>
				<artifactId>scala-maven-plugin</artifactId>
				<version>3.2.1</version>
				<executions>
					<execution>
						<id>compile-scala</id>
						<phase>compile</phase>
						<goals>
							<goal>add-source</goal>
							<goal>compile</goal>
						</goals>
					</execution>
					<execution>
						<id>test-compile-scala</id>
						<phase>test-compile</phase>
						<goals>
							<goal>add-source</goal>
							<goal>testCompile</goal>
						</goals>
					</execution>
				</executions>
				<configuration>
					<recompileMode>incremental</recompileMode>
					<scalaVersion>${scala.version}</scalaVersion>
					<args>
                        <arg>-deprecation</arg>
                    </args>
					<jvmArgs>
						<jvmArg>-Xms64m</jvmArg>
						<jvmArg>-Xmx1024m</jvmArg>
					</jvmArgs>
				</configuration>
			</plugin>

然后在main文件夹下建立scala文件夹装scala文件，scala文件夹在project structure中设置为source

scalahello文件

@RestController
class scalahello {
  @RequestMapping(value = Array("/sayscalaHello"), method = Array(RequestMethod.GET))
  def say()={
    "hello scala"
  }
}

在这里插入图片描述

整合data

		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-data-jpa</artifactId>
		</dependency>
		dependency>
			<groupId>mysql</groupId>
			<artifactId>mysql-connector-java</artifactId>
		</dependency>

2. 在这里插入图片描述
2.
pom.xml文件

server:
port: 7777
context-path: /scala-ass

spring:
datasource:
  driver-class-name: com.mysql.jdbc.Driver
  username: root
  password: root
  url: jdbc:mysql://localhost:3306/youscala0514?useSSL=false
jpa:
  hibernate:
    ddl-auto: update
  database: mysql

连接数据库成功

学会test

在这里插入图片描述
domain/MetaDatabase

package com.you0513.domain;

import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.Id;
import javax.persistence.Table;

/*
* 数据库元数据
* */
@Entity
@Table
public class MetaDatabase {
   /*数据库id
   * 名称
   * 存放的文件系统位置*/
   @Id
   @GeneratedValue
   private Integer id;
   private String name;

    public Integer getId() {
        return id;
    }

    public void setId(Integer id) {
        this.id = id;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public String getLocation() {
        return location;
    }

    public void setLocation(String location) {
        this.location = location;
    }

    private String location;
}

repository/MetaDatabaseRepository

package com.you0513.repository;

import com.you0513.domain.MetaDatabase;
import org.springframework.data.repository.CrudRepository;

public interface MetaDatabaseRepository extends CrudRepository<MetaDatabase,Integer> {
}

service/MetaDatabaseService

package com.you0513.service;

import com.you0513.domain.MetaDatabase;
import com.you0513.repository.MetaDatabaseRepository;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

import java.beans.Transient;

@Service
public class MetaDatabaseService {
    @Autowired
    private MetaDatabaseRepository metaDataBaseRepository;

    @Transactional
    public void save(MetaDatabase metaDatabase){
        metaDataBaseRepository.save(metaDatabase);
    }
    public Iterable<MetaDatabase> query(){
        return metaDataBaseRepository.findAll();
    }
}

test/service/MetaDatabaseServiceTest

package com.you0513.service;

import com.you0513.domain.MetaDatabase;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

@RunWith(SpringRunner.class)
@SpringBootTest
public class MetaDatabaseServiceTest {
    @Autowired
    private MetaDatabaseService metaDatabaseService;

    @Test
    public void testSave(){
        MetaDatabase metaDatabase=new MetaDatabase();
        metaDatabase.setName("default");
        metaDatabase.setLocation("g://");
        metaDatabaseService.save(metaDatabase);
    }

}

运行在这里插入图片描述
运行起来在mysql中自动生成了一个表，有三个字段
单元测试

查看数据库，写进去了

test/service/MetaDatabaseServiceTest再写个方法

	@Test
	public void testQuery(){
        Iterable<MetaDatabase> metaDatabases=metaDatabaseService.query();
        for (MetaDatabase metaDatabase:metaDatabases){
            System.out.println(metaDatabase.getId());
            System.out.println(metaDatabase.getName());
            System.out.println(metaDatabase.getLocation());

        }

在这里插入图片描述
运行结果

写controller

在这里插入图片描述

package com.you0513.controller;

import com.you0513.domain.MetaDatabase;
import com.you0513.service.MetaDatabaseService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("meta/database")
public class MetaDatabaseController {
    @Autowired
    private MetaDatabaseService metaDatabaseService;
    @RequestMapping(value = "/",method = RequestMethod.POST)
    public String save(@ModelAttribute MetaDatabase metaDatabase){
        metaDatabaseService.save(metaDatabase);
        return "成功";
    }
    @RequestMapping(value = "/",method = RequestMethod.GET)
    public Iterable<MetaDatabase> query(){
        return metaDatabaseService.query();
    }
}

当用户向localhost:7777/scala-ass/meta/database用post提交表单就可以写进数据库了,向localhost:7777/scala-ass/meta/database用get可以查询数据库

重构

在这里插入图片描述
ResultV0.java

package com.you0513.utils;

import java.io.Serializable;

public class ResultV0<T> implements Serializable {
    public Integer getCode() {
        return code;
    }

    public void setCode(Integer code) {
        this.code = code;
    }

    public String getMsg() {
        return msg;
    }

    public void setMsg(String msg) {
        this.msg = msg;
    }

    public T getData() {
        return data;
    }

    public void setData(T data) {
        this.data = data;
    }

    private Integer code;
    private String msg;
    private T data;
}

ResultV0Util.java

package com.you0513.utils;
/*返回值工具类*/
public class ResultV0Util {
    public  static  ResultV0 success(Object object){
        ResultV0 resultV0 = new ResultV0();
        resultV0.setData(object);
        resultV0.setCode(0);
        resultV0.setMsg("成功");

        return resultV0;
    }
    public  static  ResultV0 success(){
        return success(null);
    }

    public static  ResultV0 error(Integer code,String msg){
        ResultV0 resultV0 = new ResultV0();
        resultV0.setCode(code);
        resultV0.setMsg(msg);

        return resultV0;
    }
}

改动MetaDatabaseController.java

@RestController
@RequestMapping("meta/database")
public class MetaDatabaseController {
    @Autowired
    private MetaDatabaseService metaDatabaseService;
    @RequestMapping(value = "/",method = RequestMethod.POST)
    public ResultV0 save(@ModelAttribute MetaDatabase metaDatabase){
        metaDatabaseService.save(metaDatabase);
        return  ResultV0Util.success();
    }
    @RequestMapping(value = "/",method = RequestMethod.GET)
    public ResultV0 query(){
        return ResultV0Util.success( metaDatabaseService.query());
    }
}

管理元数据库的表用scala管理

在这里插入图片描述
MetaTable.scala

package com.you0513.domain

import javax.persistence.{Entity, GeneratedValue, Id, Table}

import scala.beans.BeanProperty

@Entity
@Table
class MetaTable {

  @Id
  @GeneratedValue
  @BeanProperty
  var id:Integer=_

  @BeanProperty
  var name:String=_

  @BeanProperty
  var tableType:String=_

  @BeanProperty
  var dbId:Integer=_
}

MetaTablRespository.scala

package com.you0513.respository

import com.you0513.domain.MetaTable
import org.springframework.data.repository.CrudRepository

trait MetaTableRespository extends CrudRepository[MetaTable,Integer]{

}

MetaTableService.scala

package com.you0513.service

import com.you0513.domain.MetaTable
import com.you0513.respository.MetaTableRespository
import org.springframework.beans.factory.annotation.Autowired
import org.springframework.stereotype.Service
import org.springframework.transaction.annotation.Transactional

@Service
class MetaTableService @Autowired()(metaTableRespository: MetaTableRespository){

  @Transactional
  def save(metaTable: MetaTable)={
    metaTableRespository.save(metaTable)
  }
  def query()={
    metaTableRespository.findAll()
  }
}

MetaTableCpntroller.scala

package com.you0513.controller

import com.you0513.domain.MetaTable
import com.you0513.service.MetaTableService
import com.you0513.utils.ResultV0Util
import org.springframework.beans.factory.annotation.Autowired
import org.springframework.web.bind.annotation.{ModelAttribute, RequestMapping, RequestMethod, ResponseBody, RestController}

@RestController
@RequestMapping(value = Array("/meta/table"))
class MetaTableCpntroller @Autowired()(metaTableService:MetaTableService){

  @RequestMapping(value=Array("/"),method = Array(RequestMethod.POST))
  def save(@ModelAttribute metaTable:MetaTable)={
    metaTableService.save(metaTable)
    ResultV0Util.success()//scala调用已有java代码
  }
  @RequestMapping(value=Array("/"),method = Array(RequestMethod.GET))
  @ResponseBody
  def  query()={
    ResultV0Util.success(metaTableService.query())//scala调用已有java代码
  }
}

当用户向localhost:7777/scala-ass/meta/table用post提交表单就可以写进数据库了,向localhost:7777/scala-ass/meta/table用get可以查询数据库