MapReducer 取前五的案列Top n

5.7 MR案例—TopN 5.7 阅读详情

1.业务需求,统计单词个数取前五Top 5

2.数据

Chief Justice Roberts, Vice President Harris, Speaker Pelosi, Leader Schumer, Leader McConnell, Vice President Pence, my distinguished guests, [and] my fellow Americans.
This is America’s day.
This is democracy’s day.
A day of history and hope.
Of renewal and resolve.
Through a crucible for the ages America has been tested anew and America has risen to the challenge.
Today, we celebrate the triumph not of a candidate, but of a cause, the cause of democracy.
The people…the will of the people has been heard and the will of the people has been heeded.
We have learned again that democracy is precious.
Democracy is fragile.
And at this hour, my friends, democracy has prevailed.
So now, on this hallowed ground where just a few days ago, violence sought to shake the Capitol’s very foundation, we come together as one nation, under God, indivisible, to carry out the peaceful transfer of power as we have for more than two centuries.
As we look ahead in our uniquely American way – restless, bold, optimistic – and set our sights on the nation we know we can be and we must be.
I thank my predecessors of both parties for their presence here today.
I thank them from the bottom of my heart.
And I know… and I know the resilience of our Constitution and the strength…the strength of our nation.
As does President Carter, who I spoke with last night but who cannot be with us today, but whom we salute for his lifetime in [of] service.
I have just taken the sacred oath each of those patriots have taken – the oath first sworn by George Washington.
But the American story depends not on any one of us, not on some of us, but on all of us.
On “We the People” who seek a more perfect Union.
This is a great nation. We are good people.
And over the centuries through storm and strife, in peace and in war, we have come so far. But we still have far to go.
We’ll press forward with speed and urgency, for we have much to do in this winter of peril and significant possibilities.
Much to repair.
Much to restore.
Much to heal.
Much to build.
And much to gain.
Few people in our nation’s history have been more challenged or found a time more challenging or difficult than the time we’re in now.
A once-in-a-century virus that silently stalks the country.
It’s taken as many lives in one year as America lost in all of World War II.
Millions of jobs have been lost.
Hundreds of thousands of businesses closed.
A cry for racial justice some 400 years in the making moves us. The dream of justice for all will be deferred no longer.
A cry for survival comes from the planet itself. A cry that can’t be any more desperate or any more clear.
And now, a rise in political extremism, white supremacy, domestic terrorism that we must confront and we will defeat.
To overcome these challenges – to restore the soul and secure the future of America – requires more than words.
It requires the most elusive of all things in a democracy:
Unity.
Unity.
In another January in Washington, on New Year’s Day 1863, Abraham Lincoln signed the Emancipation Proclamation.
When he put pen to paper, the President said, and I quote, “If my name ever goes down into history, it’ll be for this act and my whole soul is in it.”
My whole soul is in it.
Today, on this January day, my whole soul is in this:
Bringing America together.
Uniting our people.
Uniting our nation.
And I ask every American to join me in this cause.
Uniting to fight the foes we face:
Anger, resentment, and hatred.
Extremism, lawlessness, violence.
Disease, joblessness, and hopelessness.
With unity, we can do great things. Important things.
We can right wrongs.
We can put people to work in good jobs.
We can teach our children in safe schools.
We can overcome the deadly virus.
We can reward…reward work, and rebuild the middle class, and make health care secure for all.
We can deliver racial justice.
And we can make America, once again, the leading force for good in the world.
I know speaking of unity can sound to some like a foolish fantasy these days.
I know the forces that divide us are deep and they are real.
But I also know they are not new.
Our history has been a constant struggle between the American ideal that we are all created equal and the harsh, ugly reality that racism, nativism, fear, and demonization have long torn us apart.
The battle is perennial.
And victory is never assured.
Through [the] Civil War, the Great Depression, World War, 9/11, through struggle, sacrifice, and setbacks, our “better angels” have always prevailed.
In each of these moments, enough of us, enough of us have come together to carry all of us forward.
And, we can do that now.
History, faith, and reason show the way, the way of unity.
We can see each other not as adversaries but as neighbors.
We can treat each other with dignity and respect.
We can join forces, stop the shouting, and lower the temperature.
For without unity, there is no peace, only bitterness and fury.
No progress, only exhausting outrage.
No nation, only a state of chaos.
This is our historic moment of crisis and challenge, and unity is the path forward.
And, we must meet this moment as the United States of America.
If we do that, I guarantee you, we will not fail.
We have never, ever, ever failed in America when we have acted together.
And so today, at this time, [and] in this place, let’s start afresh.
All of us.
Let us begin to listen to one another again.
Hear one another.
See one another.
Show respect to one another.
Politics doesn’t have to be a raging fire destroying everything in its path.
Every disagreement doesn’t have to be a cause for total war.
And we must reject the culture in which facts themselves are manipulated and even manufactured.
My fellow Americans, we have to be different than this.
America has to be better than this.
And, I believe America is so much better than this.
Just look around.
Here we stand, in the shadow of the Capitol dome, as was mentioned earlier, completed amid the Civil War, when the Union itself was literally hanging in the balance.
Yet we endured. We prevailed.
Here we stand, looking out on the Great Mall where Dr. King spoke of his dream.
Here we stand, where 108 years ago at another inaugural, thousands of protestors tried to block brave women marching for the right to vote.
And today, we mark the swearing-in of the first woman in American history elected to national office – Vice President Kamala Harris.
Don’t tell me things can’t change.
Here we stand across the Potomac from Arlington Cemetery, where heroes who gave the last full measure of devotion rest in eternal peace.
And here we stand, just days after a riotous mob thought they could use violence to silence the will of the people, to stop the work of our democracy, to drive us from this sacred ground.
That did not happen.
It will never happen.
Not today.
Not tomorrow.
Not ever.
Not ever.
To all those who supported our campaign, I’m humbled by the faith you have placed in us.
To all those who did not support us, let me say this: Hear me out as we move forward. Take a measure of me and my heart.
And if you still disagree, so be it.
That’s democracy. That’s America. The right to dissent peaceably, within the guardrails of our Republic, is perhaps this nation’s greatest strength.
Yet hear me clearly: Disagreement must not lead to disunion.
And I pledge this to you: I will be a President for all Americans. All Americans.
And I promise you: I will fight as hard for those who did not support me as for those who did.
Many centuries ago, Saint Augustine, a saint of my church, wrote that a people was a multitude defined by the common objects of their love – defined by the common objects of their love.
What are the common objects we as Americans love that define us as Americans?
I think we know.
Opportunity.
Security.
Liberty.
Dignity.
Respect.
Honor.
And, yes, the truth.
Recent weeks and months have taught us a painful lesson.
There is truth and there are lies.
Lies told for power and for profit.
And each of us has a duty and responsibility, as citizens, as Americans, and especially as leaders – leaders who have pledged to honor our Constitution and protect our nation – to defend the truth and defeat the lies.
Look, I understand that many fellow Americans view the future with fear and trepidation.
I understand they worry about their jobs. I understand, like my dad, they lay in bed staring at night, staring at the ceiling, wondering, “Can I keep my health care? Can I pay my mortgage?” Thinking about their families, about what comes next.
I promise you: I get it.
But the answer is not to turn inward, to retreat into competing factions, distrusting those who don’t look like…look like you, or worship the way you do, or don’t get their news from the same sources you do.
We must end this uncivil war that pits red against blue, rural versus urban, conservative versus liberal.
We can do this if we open our souls instead of hardening our hearts.
If we show a little tolerance and humility.
And if we’re willing to stand in the other person’s shoes, as my mom would say, “just for a moment, stand in their shoes.”
Because here is the thing about life: There is no accounting for what fate will deal you.
Some days when you need a hand.
There are other days when we’re called to lend a hand.
That’s how it has to be. That’s what we do for one another.
And, if we are this way, our country will be stronger, more prosperous, more ready for the future. And we can still disagree.
My fellow Americans, in the work ahead of us, we’re going to need each other.
We need all our strength to persevere through this dark winter.
We are entering what may be the toughest and deadliest period of the virus.
We must set aside politics and finally face this pandemic as one nation. One nation.
And I promise you this: as the Bible says weeping may endure for a night but joy cometh in the morning.
We will get through this, together. Together.
Look, folks, all my colleagues I served with in the House and the Senate up here, we all understand the world is watching, watching all of us today.
So here is my message to those beyond our borders: America has been tested and we have come out stronger for it.
We will repair our alliances and engage with the world once again.
Not to meet yesterday’s challenges, but today’s and tomorrow’s challenges.
And we will lead not merely by the example of our power but by the power of our example.
We’ll be a strong and trusted partner for peace, progress, and security.
Look, you all know we’ve been through so much in this nation.
And [in] my first act as President, I’d like to ask you to join me in a moment of silent prayer to remember all those who we lost in this past year to the pandemic.
Those 400,000 fellow Americans – moms, dads, husbands, wives, sons, daughters, friends, neighbors, and co-workers.
We’ll honor them by becoming the people and the nation we know we can and should be.
So I ask you, let’s say a silent prayer for those who’ve lost their lives and those left behind, and for our country.
Amen.
Folks, this is a time of testing.
We face an attack on our democracy and on truth.
A raging virus.
Growing inequity.
The sting of systemic racism.
A climate in crisis.
America’s role in the world.
Any one of these would be enough to challenge us in profound ways.
But the fact is we face them all at once, presenting this nation with one of the gravest of responsibilities we’ve had.
Now we’re going to be tested. Are we going to step up, all of us?
It is a time for boldness, for there is so much to do.
And, this is certain.
I promise you: we will be judged, you and I, by how we resolve these cascading crises of our era.
We will rise to the occasion – here’s the question.
Will we master this rare and difficult hour?
Will we meet our obligations and pass along a new and better world to our children?
I believe we must. I’m sure you do as well. I believe we will.
And when we do, we’ll write the next great chapter in the history of the United States of America, the American story, a story that might sound something like a song that means a lot to me. It’s called “American Anthem” and there’s one verse that stands out at least for me, and it goes like this:
“The work and prayers
Of centuries have brought us to this day
What shall be our legacy?
What will our children say?
…
Let me know in my heart
When my days are through
America
America
I gave my best to you.”
Let’s add. Let us add our own work and prayers to the unfolding story of our great nation.
If we do this, then when our days are through, our children and our children’s children will say of us: “They gave their best. They did their duty. They healed a broken land.”
My fellow Americans, I close today where I began, with a sacred oath.
Before God and all of you, I give you my word.
I will always level with you.
I will defend the Constitution.
I’ll defend our democracy.
I’ll defend America.
And I’ll give all, all of you, keep everything I do in your service thinking not of power, but of possibilities.
Not of personal interest, but [of] the public good.
And together, we shall write an American story of hope, not fear.
Of unity, not division.
Of light, not darkness.
A story of decency and dignity.
Love and healing.
Greatness and goodness.
May this be the story that guides us.
The story that inspires us.
And the story that tells ages yet to come that we answered the call of history.
We met the moment.
Democracy and hope, truth and justice, did not die on our watch but thrived.
That America secured liberty at home and stood once again as a beacon to the world.
That is what we owe our forebearers, one another, and generations to follow.
So, with purpose and resolve, we turn to those tasks of our time.
Sustained by faith.
Driven by conviction.
And, devoted to one another and the country we love with all our hearts.
May God bless America and may God protect our troops.
Thank you, America.

3.创建maven项目

4.需要先对上面数据进行统计,统计个数,就是写一个wordcount

4.导入依赖

<dependencies>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-client</artifactId>
            <version>3.1.3</version>
        </dependency>

        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
        </dependency>

        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-log4j12</artifactId>
            <version>1.7.30</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.6.1</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>

            <plugin>
                <artifactId>maven-assembly-plugin</artifactId>
                <configuration>
                    <descriptorRefs>
                        <descriptorRef>jar-with-dependencies</descriptorRef>
                    </descriptorRefs>
                </configuration>
                <executions>
                    <execution>
                        <id>make-assembly</id>
                        <phase>package</phase>
                        <goals>
                            <goal>single</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

4.1,mapper

package com.lhh;

import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Mapper;

import java.io.IOException;

/**
 * @program: lhh
 * @description:
 * @author: 华仔
 * @create: 2021-04-16 09:29
 */
public class KSMapper extends Mapper<LongWritable, Text,Text, IntWritable> {
    @Override
    protected void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {
       //读取一行
        String stxt = value.toString();
        //替换标点符号为 \\s  \\s正则表达式,表示一个或多个空格
        String svalue = stxt.replaceAll("\\W"," ");
        //根据 \\s拆分
        String[] split = svalue.split("\\s+");
        for (String s : split) {  //循环写出
            context.write(new Text(s),new IntWritable(1));
        }
    }
}


4.2reducer

package com.lhh;

import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Reducer;

import java.io.IOException;


/**
 * @program: lhh
 * @description:
 * @author: 华仔
 * @create: 2021-04-16 09:29
 */
public class KSReduce extends Reducer<Text, IntWritable, Text, IntWritable> {
    @Override
    protected void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException {
        //声明变量
        int num=0; 
        for (IntWritable value : values) {
            num += value.get(); //累加
        }
        //写出数据
        context.write(key,new IntWritable(num));
    }
}

4.3job

package com.lhh;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;

import java.io.IOException;

/**
 * @program: lhh
 * @description:
 * @author: 华仔
 * @create: 2021-04-16 09:29
 */
public class KSJob {
    public static void main(String[] args) throws IOException, ClassNotFoundException, InterruptedException {
        //获取job
        Configuration conf = new Configuration();
        Job job = Job.getInstance(conf);
        job.setJarByClass(KSJob.class);
        //设置map reduce
        job.setMapperClass(KSMapper.class);
        job.setReducerClass(KSReduce.class);

        //设置map输出类
        job.setMapOutputKeyClass(Text.class);
        job.setMapOutputValueClass(IntWritable.class);

        //设置最终输出类
        job.setOutputKeyClass(Text.class);
        job.setOutputValueClass(IntWritable.class);

        //设置输入和输出路径
        FileInputFormat.setInputPaths(job, new Path("D:/test/in/ceshi.txt"));
        FileOutputFormat.setOutputPath(job, new Path("D:/test/out/ceshiout1"));

        //等待任务完成
        boolean b = job.waitForCompletion(true);
        if(b){
            System.out.println("程序完成");
        }else{
            System.out.println("程序异常");
        }
        //关闭程序
        System.exit(b ? 0 : 1);
    }
}

注意:这里是将上面的结果作为这一次的数据

5.导入依赖

<dependencies>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-client</artifactId>
            <version>3.1.3</version>
        </dependency>

        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
        </dependency>

        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-log4j12</artifactId>
            <version>1.7.30</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.6.1</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>

            <plugin>
                <artifactId>maven-assembly-plugin</artifactId>
                <configuration>
                    <descriptorRefs>
                        <descriptorRef>jar-with-dependencies</descriptorRef>
                    </descriptorRefs>
                </configuration>
                <executions>
                    <execution>
                        <id>make-assembly</id>
                        <phase>package</phase>
                        <goals>
                            <goal>single</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

5.1.创建Bean

package lhh;

import org.apache.hadoop.io.WritableComparable;

import java.io.DataInput;
import java.io.DataOutput;
import java.io.IOException;

/**
 * @program: lhh
 * @description:
 * @author: 华仔
 * @create: 2021-03-09 14:14
 */
public class KSBean implements WritableComparable<KSBean> {

    //创建变量并且set get and toString
    private String word;

    private int num;

    private int sort;
    public KSBean() {
    }

    public KSBean(String word, int num, int sort) {
        this.word = word;
        this.num = num;
        this.sort = sort;
    }

    @Override
    public String toString() {
        return word + '\t' + num +'\t'+ sort;
    }




    //实行比较器,指定排序规则
    public int compareTo(KSBean o) {
        //降序排序,升序排序就换个位置
        if(o.getNum()>this.getNum()){ //如果相等就返回1,不相同就返回-1
            return 1;
        }else{
            return -1;
        }
    }

    //实现序列化
    public void write(DataOutput dataOutput) throws IOException {
        dataOutput.writeUTF(word);
        dataOutput.writeInt(num);
    }

    //反序列化
    public void readFields(DataInput dataInput) throws IOException {
        this.word = dataInput.readUTF(); //读取字符串
        this.num = dataInput.readInt();//读取数字
    }

    public String getWord() {
        return word;
    }

    public void setWord(String word) {
        this.word = word;
    }

    public int getNum() {
        return num;
    }

    public void setNum(int num) {
        this.num = num;
    }

    public int getSort() {
        return sort;
    }

    public void setSort(int sort) {
        this.sort = sort;
    }
}

5.2.创建Mapper

package lhh;

import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Mapper;

import java.io.IOException;

/**
 * @program: lhh
 * @description:
 * @author: 华仔
 * @create: 2021-04-16 09:29
 */
public class KSMapper extends Mapper<LongWritable, Text,KSBean, Text> {
    @Override
    protected void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {
       //读取一行
        String stxt = value.toString();
        String[] split = stxt.split("\t");
        String str1=split[0]; //取值
        String str2=split[1];
        KSBean ksBean = new KSBean(); //
        ksBean.setWord(str1);
        ksBean.setNum(Integer.parseInt(str2));
        context.write(ksBean,new Text(ksBean.getWord()));
    }


}


5…3reducer 序号不跳序 1 2 3 4 5 ,意思就如果有数字一样的,还是直接排下去,方式一

package lhh;

import org.apache.hadoop.io.NullWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Reducer;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

/**
 * @program: lhh
 * @description:
 * @author: 华仔
 * @create: 2021-04-16 09:29
 */
public class KSReducebutiaoxu extends Reducer<KSBean, Text, NullWritable, KSBean> {


    int sum = 0; //正常的排名
    List<KSBean> list = new ArrayList<>();
    KSBean ksBean = null;
    int rank = 0; //计算上一次的排名

    int count = 0; //计算上一次数量


    @Override
    protected void reduce(KSBean key, Iterable<Text> values, Context context) throws IOException, InterruptedException {
        for (Text value : values) {
            //当记录和上一次相等时
            if (key.getNum() == count) {
                ksBean = new KSBean(key.getWord(), key.getNum(), rank);
                list.add(ksBean);
            } else {
                if (rank >= 5) {
                    break;
                } else {
                    rank++;
                    count = key.getNum();
                    ksBean = new KSBean(key.getWord(), key.getNum(), rank);
                    list.add(ksBean);
                }
            }
        }
    }

    //只执行一次
    @Override
    protected void cleanup(Context context) throws IOException, InterruptedException {
        for (KSBean bean : list) {
            context.write(NullWritable.get(),bean);
        }
    }

}

5.4reducer 跳序 1 2 2 4 5 意思就如果有数字一样的,就会跳过序号,方式二

package lhh;

import org.apache.hadoop.io.NullWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Reducer;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

/**
 * @program: lhh
 * @description:
 * @author: 华仔
 * @create: 2021-04-18 16:19
 */
public class KSReducetiaoxu extends Reducer<KSBean, Text, NullWritable, KSBean>{

    int sum = 0; //正常的排名
    List<KSBean> list = new ArrayList<>();
    KSBean ksBean = null;
    int rank = 0; //计算上一次的排名

    int count = 0; //计算上一次数量


    @Override
    protected void reduce(KSBean key, Iterable<Text> values, Context context) throws IOException, InterruptedException {
        for (Text value : values) {
            //当记录和上一次相等时
            if (key.getNum() == count) {
                ksBean = new KSBean(key.getWord(), key.getNum(), rank);
                list.add(ksBean);
                sum++;
            } else {
                if (sum >= 5) {
                    break;
                } else {
                    sum++;
                    rank = sum;
                    count = key.getNum();
                    ksBean = new KSBean(key.getWord(), key.getNum(), rank);
                    list.add(ksBean);
                }
            }
        }
    }

    @Override
    protected void cleanup(Reducer.Context context) throws IOException, InterruptedException {
        for (KSBean bean : list) {
            context.write(NullWritable.get(),bean);
        }
    }

}

5.5job

package lhh;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.NullWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;

import java.io.IOException;

/**
 * @program: lhh
 * @description:
 * @author: 华仔
 * @create: 2021-04-16 09:29
 */
public class KSJob {
    public static void main(String[] args) throws IOException, ClassNotFoundException, InterruptedException {
        //获取job
        Configuration conf = new Configuration();
        Job job = Job.getInstance(conf);
        job.setJarByClass(KSJob.class);
        //设置map reduce
        job.setMapperClass(KSMapper.class);
        job.setReducerClass(KSReducebutiaoxu.class);

        //设置map输出类
        job.setMapOutputKeyClass(KSBean.class);
        job.setMapOutputValueClass(Text.class);

        //设置最终输出类
        job.setOutputKeyClass(NullWritable.class);
        job.setOutputValueClass(KSBean.class);

        //设置输入和输出路径
        FileInputFormat.setInputPaths(job, new Path("D:/test/out/test/abc.txt"));
        FileOutputFormat.setOutputPath(job, new Path("D:/test/out/abc6"));

        //等待任务完成
        boolean b = job.waitForCompletion(true);
        if(b){
            System.out.println("程序完成");
        }else{
            System.out.println("程序异常");
        }
        //关闭程序
        System.exit(b ? 0 : 1);
    }
}

ELK 企业级日志分析系统(1) ELK平台是一套完整的日志集中处理解决方案,将、Logstash和Kiabana三个开源工具配合使用, 完成更强大的用户对日志的查询、排序、统计需求。ElasticSearch:是基于Lucene(一个全文检索引擎的架构)开发的分布式存储检索引擎,用来存储各类日志。Elasticsearch 是用 Java 开发的,可通过 RESTful Web 接口,让用户可以通过浏览器与 Elasticsearch 通信。 阅读详情

相关推荐

ECU-TEST 快速入门

ECU-TEST 是由位于德国德累斯顿的TraceTronic公司开发的一款用于嵌入式系统测试验证软件工具。自从2003年首次发布ECU-TEST ,该软件成为了汽车ECU开发的标准工具 ,同时也逐步成为了重型机械和工业自动化开发的标准工具 。该软件是起源自于一个控制单元的系统化测试的项目研究,从而为从德累斯顿工业大学脱离出来的TraceTronic公司奠定了基础。ECU-TEST旨在完成测试案例的规范化、评估、求解和归档。得益于该软件包含大量的自动化测试方法使得测试案例的所有必要功能如创建、求解和评估

wangying202的博客 1万+

Map集合遍历的5种方法

TreeMap<Integer,String> map=new TreeMap(); map.put(1416270241, "东皇太一"); map.put(1416270243, "不知火舞"); map.put(1416270244, "伽罗"); map.put(1416270214, "东方不败"); System.out.println("女娲"); Set&...

空山新雨后,天气晚来秋 1345

软考29-上午题-【数据结构】-排序

数据结构7种排序

qq_31532983的博客 1581

Java 对Map进行排序,并取出前几个值

1、创建一个测试用的Map Map<String, Integer> map = new HashMap<>(); map.put("张三",15); map.put("李四",13); map.put("王二麻",99); map.put("甲一",28); map.put("丙二",9); System.out.println(sortMap(map)); 2、降序排

qq_39486119的博客 12万+

CANape快速上手

文章帮助记录CANape学习过程。

小弱鸡的博客 1万+

从零开始:H20服务器上DeepSeek R1 671B大模型部署与压力测试全攻略

最近,我有幸在工作中接触到了DeepSeek R1 671B模型,这是目前中文开源领域参数量最大的高质量模型之一。DeepSeek团队在2024年推出的这款模型,以其惊人的6710亿参数量和出色的推理性能,引起了业界广泛关注。作为一名AI基础设施工程师,我有机会在H20服务器上部署这个庞然大物,并对其进行了全面的压力测试。这篇文章将详细记录我的部署过程和性能测试方法,希望能为大家提供一些参考。💡为什么选择DeepSeek R1?超大规模参数量(671B)优秀的中英文理解能力开源可商用的许可证。

wangli的博客 1万+

Scala实现Mapreduce程序2-----Top5

输入n个数,返回TOP5的数字 scala实现,以各个数字为key,""为空,按照key进行排序,取出前5个 object Top5 { def main(args: Array[String]): Unit = { val conf = new SparkConf().setMaster("") val sc = new SparkContext(conf) v...

weixin_30343157的博客 248

实战分享:一文读懂RS-232总线

RS-232标准接口(又称EIARS-232)是常用的串行通信接口标准之一,它是由美国电子工业协会(Electronic IndustryAssociation,EIA)联合贝尔系统公司、调制解调器厂家及计算机终端生产厂家于1970年共同制定,其全名是“数据终端设备(DTE)和数据通信设备(DCE)之间串行二进制数据交换接口技术标准”。

阿荣嵌入式 8527

MapReduce读取文本,实现降序排序

目录 1、Maven导入hadoop-client包 2、core-site.xml文件配置 3、log4j.properties 文件配置 4、Top5.java(主要代码) 5、测试数据 6、运行结果 1、Maven导入hadoop-client包 <dependency> <groupId>org.apache.hadoop</gro...

星辰大数据的博客 2536

泛微E8流程监控指南:如何用SQL实时追踪审批节点状态?

本文详细介绍了如何利用SQL实时监控泛微E8系统中的审批流程状态,包括核心数据表结构解析、全链路监控SQL实战以及智能预警机制设计。通过数据库直接查询和状态关联分析,帮助企业快速定位卡顿节点,提升流程效率。适用于需要优化OA系统流程管理的企业IT人员和业务主管。

weixin_30687051的博客 440

UCIe物理层 —— 逻辑协议 (2)

本章开始主要介绍UCIe协议第四章,物理层逻辑定义和规范。如其他的通信协议一样,接口的物理层包含两个部分,逻辑子层和电气子层。其中逻辑层主要负责:1.定义接口信号和时序2. 定义发送端如何将上层数据分发到各个物理通道以及接收端如何把从各个通道收到的数据恢复成byte stream(Flit)。3. Broken Lane的repair和remap。4. 链路初始化,训练(training)和电源状态管理/切换。5. 扰码Scrambling和training sequence的产生。

m0_47581558的博客 1483

Hadoop之MapReduce程序应用一

摘要:MapReduce程序处理专利数据集。 关键词:MapReduce程序 专利数据集

数据人 3874

1. 动手开发第一个 SAP CDS View

在开始学习前,大家需要根据这篇文章介绍的步骤,在自己本地电脑上,把 ABAP Development Tool 安装好。右键单击这个添加好的 ABAP 开发包,选择新建一个 ABAP Repository Object,类型就选择。,就可以像在 SAPGUI 里使用 SE16 那样,查看这个 CDS View 的内容。笔者下一篇文章,会详细介绍我们创建的第一个 CDS View 的每一行代码的详细含义。从本文开始,正式进入 CDS View 开发的学习。的条件,然后点击 Run 重新执行,得到结果。

2007 年 ~ 2025 年,深耕 SAP 技术 18 年 1274

MapReduce案例(数据中获取最大值TopN)

案例: 案列: data.txt 10 9 8 7 6 5 1 2 3 4 11 12 13 14 15 20 19 18 17 16 package squencefile; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.Path; import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Null...

有的话没说出来之前,你是他的主人,一旦说出来你就成了他的奴隶||想要干好事,别太把自己当人,别把别人太不当人||认识这个人就是开了一扇窗户,就能看到不一样的东西,听到不一样的声音,能让你思考、觉悟,这已经够了 1535

LinePhone3.2.7(android端最新版本) sip客户端

SIP客户端For android:LinePhone3.2.7(最新版本),官网地址:http://www.linphone.org/

VC中sqlite的使用

在VC中对sqlite的使用,实现数据库大批数据的快速插入,读取数据库中的数据在ListCtrl中显示

对map根据value排序

1.排序并取出前五个 public static void main(String[] args) throws Exception{ Map<String, Integer> hashMap = new HashMap<String, Integer>(); hashMap.put("1", 1); hashMap.put("2", 2); hashMa...

qianhuan_的博客 970

华为的C语言编程规范.pdf

华为技术有限公司内部技术规范。本规范制定了编写C语言程序的基本原则、规则和建议。从代码的清晰、简洁、可测试、安全、程序效率、可移植各个方面对C语言编程作出了具体指导。

MapReduce编程

实验目的(转载地址)通过实验掌握基本的MapReduce编程方法。掌握用MapReduce解决一些常见的数据处理问题,包括数据去重、数据排序和数据挖掘等。通过操作MapReduce的实验,模仿实验内容,深入理解MapReduce的过程,熟悉MapReduce程序的编程方式。实验平台操作系统:Ubuntu-16.04Hadoop版本:2.6.0JDK版本:1.8IDE:Eclipse实验内容和要求一...

panjiao119的博客 3149

大中型超市收银系统(可直接运行)

光步前台收银系统系统功能:商品销售、商品退货、挂单、收银员权限、会员销售、单品取消、整单取消、商品查询、脱机销售、脱机数据上传,基本资料下载,开钱箱、小票打印、收款结算,支持多币种付款方式(现金、储值卡、礼券、微信等),支持多种促销。运行环境:.net 4.0登陆信息用户名:1001 密码:123456内置商品信息,商品编码可输入00001000 - 00010000,测试效果相关功能键F5 结算 F9挂单 F11商品查询 F8退货 Insert单品取消 Del整单取消 F12开钱箱F2 会员 Ctrl+Z退出系统 Home 锁屏 相关源码待公开

MapReduce设计模式:求最小值、最大值和计数

MapReduce设计模式:求最小值、最大值和计数的MapReduce设计模型如下: MinMaxCount  类如下: package com; import java.io.IOException; import java.text.ParseException; import java.text.SimpleDateFormat; import java.util.Date;

fengshi_fengshi的专栏 1768

【hadoop学习之路】MapReduce实现数字排序

参考视频:【狂野大数据】一天搞定大数据之MapReduce 目录 1. MapReduce 运行流程 2. 规划自定义的 MapReduce 程序 3. 代码实现 1. MapReduce 运行流程 MapReduce主要经过三个阶段: ① Map阶段 1)以<k1, v1>格式读取文本文件,其中k1为文本偏移量,通常定义为数字类型(如LongWritable),v2为文本,通常定义为Text类型 2)自定义语句,用于处理<k1, v1>及生成<k2, v.

onion23的博客 4372

使用 MapReduce 实现分组排名

题目: 1.输出男女组各前3名。 2.分组降序输出所有人的成绩。 3.对应的SQL语句。 数据: scores: name,age,gender,score 解答如下: 1. 输出男女组各前3名。 Java代码 1 class ContestStatTop3Mapper extends Mapper { 2 3 private Text outKey = new Text();

cauwt的博客 1085

【MapReduce】扩展案例 ---- TopN(求取数据集前N名)

文章目录TopN(求取数据集前N名)需求需求分析代码实现① Bean类② Mapper 阶段③ Reducer 阶段④ Driver 阶段总结 TopN(求取数据集前N名) 需求 对之前统计的上、下行流量总值输出结果进行加工,输出流量使用量在前10的用户信息 (1)输入数据 (2)输出数据 返回顶部 需求分析 返回顶部 代码实现 ① Bean类 创建Bean类,封装上下行、总流量对象 创建无参构造方法 — 供反序列化使用 重写序列化、反序列化方法 重写toString()方法 继承自Wri..

懂得一千零一种,赋予你失败的方法! 1512

【R语言】数据框排序并取前n行或者前n列

## 排序 ## 默认df为某数据框 df<-arrange(df,字段名1,,字段名2...) #支持多字段排序(越在前的字段优先级越高),默认为升序,若需降序则在字段名前加负号,即arrange(数据框,-字段名) ## 取前10行数据 df<-df[1:10,] ## 取前10列数据 df<-df[,1:10] ...

weixin_41673714的博客 2万+

map分组后取前10个_map根据属性排序、取出map前n个

/*** map根据value排序* flag = 1 正序* flag = 0 倒序** @param map* @param flag* @return*/public static > LinkedHashMap sortByValue(Map map, int flag) {LinkedHashMap sortMap = new LinkedHashMap<>();if ...

weixin_32613887的博客 3312

Map按value排序并取前10条

//Map 按value值从大到小排序,并取前10 public static Map sortMapByValue(Map map) {  Map sortedMap = new LinkedHashMap();  List>lists=new ArrayList>(map.entrySet()); Collections.sort(lists,new Comparator>()

Vicky_ZWH的博客 1万+

JAVA中对list map根据map某个key值进行排序并取出前四条数据

import java.util.ArrayList;   import java.util.Collections; import java.util.Comparator; import java.util.HashMap; import java.util.List; import java.util.Map;   public class Test {     public static...

qacjava的博客 8644

Mapreduce的排序

1. Mapreduce排序 排序是MapReduce的灵魂,MapReduce在Map和Reduce的两个阶段当中,都在反复地执行排序。 1.1 全局排序 排序分为全局排序、部分排序、二次排序、辅助排序。全局排序,就是在一个MapReduce程序产生的输出文件中,所有的结果都是按照某个策略进行排序的,例如降序还是升序。MapReduce只能保证一个分区内的数据是key有序的,一个分区对应一个reduce,因此只有一个reduce就保证了数据全局有序,但是这样又不能用到Hadoop集群的优势。 1.

L13763338360的博客 4859

MapReduce实现分组排序

MapReduce实现分组排序 以某次竞赛为例,分别进行如果实现: 取每组中男生前三名成绩和女生前三名成绩按照年龄分组降序输出所有人的成绩等价的SQL 0. 预备知识 0.1 基于MapReduce实现分组、排序: 分组:相当于group by。MapReduce的实现:相当于分区,以求处理手机上网日志为例,把手机号和非手机号分为两组。

谦蓦的博客 4773
上一篇: HADOOP MapReduce WordCount案列
下一篇: 1.错误解决 Error GC overhead limit exceeded
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

热心市民爱抽烟屁

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值