avro gradle plugin sample usage avro gradle plugin sample usage hadoop hadoop

avro gradle plugin sample usage


I figured out how to do it myself. The following is a snippet that I would like to share for people who might run into the same issues as I did:

apply plugin: 'java'apply plugin: 'avro-gradle-plugin'sourceCompatibility = "1.6"targetCompatibility = "1.6"buildscript {  repositories {    maven {       // your maven repo information here    }  }  dependencies {    classpath 'org.apache.maven:maven-artifact:2.2.1'    classpath 'org.apache.avro:avro-compiler:1.7.1'    classpath 'org.apache.avro.gradle:avro-gradle-plugin:1.7.1'  }}compileAvro.source = 'src/main/avro'compileAvro.destinationDir = file("$buildDir/generated-sources/avro")sourceSets {  main {    java {      srcDir compileAvro.destinationDir    }  }}dependencies {  compileAvro}


I found "com.commercehub.gradle.plugin.avro" gradle plugin to work better.

use the folllowing:

// Gradle 2.1 and laterplugins {  id "com.commercehub.gradle.plugin.avro" version "VERSION"}// Earlier versions of Gradlebuildscript {    repositories {        jcenter()    }    dependencies {        classpath "com.commercehub.gradle.plugin:gradle-avro-plugin:VERSION"    }}apply plugin: "com.commercehub.gradle.plugin.avro"

more details at https://github.com/commercehub-oss/gradle-avro-plugin


When evaluating a plugin the following questions needs to be asked:

  • Are generated files included into source jar?
  • Is plugin fast? Good plugin use avro tools api instead of forking VM for every file. For large amount of files creating VM for every file can take 10min to compile.
  • Do you need intermediate avsc files?
  • Is build incremental (i.e. do not regenerate all files unless one of the sources changed)?
  • Is plugin flexible enough to give access to generated schema files, so further actions, such as registration schema in schema repository can be made?

It is easy enough to implement without any plugin if you are not happy with plugin or need more flexibility.

//// define source and destination//def avdlFiles = fileTree('src/Schemas').include('**/*.avdl')// Do NOT generate into $buildDir, because IntelliJ will ignore files in // this location and will show errors in source codedef generatedJavaDir = "generated/avro/java"sourceSets.main.java.srcDir generatedJavaDir//// Make avro-tools available to the build script//buildscript {    dependencies {        classpath group:'org.apache.avro', name:'avro-tools' ,version: avro_version    }}//// Define task's input and output, compile idl to schema and schema to java//task buildAvroDtos(){    group = "build"    inputs.files avdlFiles    outputs.dir generatedJavaDir    doLast{        avdlFiles.each { avdlFile ->            def parser = new org.apache.avro.compiler.idl.Idl(avdlFile)            parser.CompilationUnit().getTypes().each { schema ->                def compiler = new org.apache.avro.compiler.specific.SpecificCompiler(schema)                compiler.compileToDestination(avdlFile, new File(generatedJavaDir))            }        }    }}//// Publish source jar, including generated files//task sourceJar(type: Jar, dependsOn: buildAvroDtos) {    from sourceSets.main.allSource    // Package schemas into source jar    into("Schemas") { from avdlFiles }}// Clean "generated" folder upon "clean" taskclean {    delete('generated')}