Open-Source Large Vocabulary CSR Engine Julius

最新Open-vocabulary方法个人学习小结 open-vocabulary最新方法的学习小结 阅读详情
top
  • What's New?

    About Julius

    "Julius" is a high-performance, two-pass large vocabularycontinuous speech recognition (LVCSR) decoder software forspeech-related researchers and developers. Based on word N-gram andcontext-dependent HMM, it can perform almost real-time decoding onmost current PCs in 60k word dictation task. Major search techniquesare fully incorporated such as tree lexicon, N-gram factoring,cross-word context dependency handling, enveloped beam search,Gaussian pruning, Gaussian selection, etc. Besides search efficiency,it is also modularized carefully to be independent from modelstructures, and various HMM types are supported such as shared-statetriphones and tied-mixture models, with any number of mixtures,states, or phones. Standard formats are adopted to cope with otherfree modeling toolkit such as HTK, CMU-Cam SLM toolkit, etc.

    The main platform is Linux and other Unix workstations, and also workson Windows. Most recent version is developed on Linux and Windows(cygwin / mingw), and also has Microsoft SAPI version. Julius isdistributed with open license together with source codes.

    Julius has been developed as a research software for Japanese LVCSRsince 1997, and the work was continued under IPA Japanese dictationtoolkit project (1997-2000), Continuous Speech RecognitionConsortium, Japan (CSRC) (2000-2003) and currently Interactive Speech TechnologyConsortium (ISTC).

    Features

    • An open-source software (see terms and conditions of license)
    • Real-time, hi-speed, accurate recognition based on 2-pass strategy.
    • Low memory requirement: less than 32MBytes required for work area (<64MBytes for 20k-word dictation with on-memory 3-gram LM).
    • Supports LM of N-gram, grammar, and isolated word.
    • Language and unit-dependent: Any LM in ARPA standard format and AM in HTK ascii hmmdefs format can be used.
    • Highly configurable: can set various search parameters. Alsoalternate decoding algorithm (1-best/word-pair approx., wordtrellis/word graph intermediates, etc.) can be chosen.
    • Full source code documentation and manual in Engligh / Japanese.
    • List of major supported features:
      • On-the-fly recognition for microphone and network input
      • GMM-based input rejection
      • Successive decoding, delimiting input by short pauses
      • N-best output
      • Word graph output
      • Forced alignment on word, phoneme, and state level
      • Confidence scoring
      • Server mode and control API
      • Many search parameters for tuning its performance
      • Character code conversion for result output.
      • (Rev. 4) Engine becomes Library and offers simple API
      • (Rev. 4) Long N-gram support
      • (Rev. 4) Run with forward / backward N-gram only
      • (Rev. 4) Confusion network output
      • (Rev. 4) Arbitrary multi-model decoding in a single thread.
      • (Rev. 4) Rapid isolated word recognition
      • (Rev. 4) User-defined LM function embedding

    Contact

    For any questions, e-mail to julius-info atlists.sourceforge.jp.

    The chief developer and maintainer of Julius (Unix) is LEE Akinobu(ri at nitech.ac.jp).

    A forum has been opened. Please post questions, look for information, or share knowledges in Julius forum.


    Latest version: 4.3.1

    The latest version is 4.3.1, released on January 15, 2014.

    Version 4.3.1 is a bug fix release. Several bugs has been fixed.

    See the "Release.txt" file for the full list of updates.
    Run with "-help" to see full list of options.


    Download Julius

    Note: you should prepare a language model and an acoustic model to run a speech recognition with Julius, See About Models below.

    Get current version

    You can get old version from Development site.

    Get the latest codes via CVS

    You can get the current snapshot of source tree via anonymous CVS:
    cvs -z3 -d:pserver:anonymous@cvs.sourceforge.jp:/cvsroot/julius co julius4
    
    Please note that current CVS repository has moved to "julius4" instead of "julius".You can also receive update notices by subscribing to julius-cvs@lists.sourceforge.jp.Messages will be sent each time the source has been changed on theCVS. Anyone can subscribe from julius-cvs@lists.sourceforge.jpmanagement page.

    Get Julius for Windows SAPI

    Julius for SAPI is MS Windows version of Julius/Julian which implementsMicrosoft(R) Speech API (SAPI) 5.1.You can use this version of Julius as a SAPI Voice Recognizer inapplications created for SAPI (e.g. Office XP).

    The recent version is fully SAPI-5.1 compliant, and it also supportsSALT extension.

    Julius for SAPI assumes that the user language and the application'sgrammar is in Japanese. So it is a little troublesome in case of theother languages because Julius for SAPI does not know thepronunciation of the words in a grammar. If you define pronunciationsto each of these, it may work, but we have not tried it.

    Please read following documents for detail.

    Download: you should install both of them. (last updated: 2004/02/05 for ver. 2.3)

    Related Tools

    word / phoneme segmentation kit

    This toolkit helps performing "forced alignment" with speechrecognition engine Julius with grammar-based recognition.This kit uses Julius to do forced alignment to a speech fileby generating grammar for each samples from transcription.

    HTK-to-Julius grammar converter

    This toolkit converts an HTK recognition grammar into Julian format.A word network (SLF) will be converted to DFA format, and the wordsin the SLF are extracted from the dictionary to be used in Julian.Furthermore, word category will be automatically detected and definedto optimize performance in Julian.

    About Models

    Since Julius itself is a language-independent decoding program, youcan make a recognizer of a language if given an appropriate languagemodel and acoustic model for the target language. The recognition accuracylargely depends on the models.

    Julius adoptsacoustic models in HTK ascii format, pronunciation dictionary inalmost HTK format, and word 3-gram language models in ARPA standardformat (forward 2-gram and reverse 3-gram trained from same corpus).

    We had already examined English dictations with Julius, and anotherresearcher has reported that Julius has also worked well in English,Slovenian (see pp.681--684 of Proc. ICSLP2002), French, Thai language,and many other Languages.

    Here you can get Japanese and English free language/acoustic models.

    If you have any language or acoustic model that can be distributed as a freeware, would you please contact us?We want to run dictation kit on various languages other than Japanese,and share them freely to provide a free speech recognition systemavailable for various languages.


    Documents and Notes

    Documentation

    We are also making a complete documentation of Julius, fully updatedfor the current version. The document is called "Juliusbook", and its initial release has been done in Japanese. We are now making English version.

    How to write a grammar for Julius

    The format of recognition grammar for Julius is briefly described here.

    References


    ChangeLog

    4.3.1 (2014.1.15)
    Fixed bugs:
    - Compilation error on OS X.
    - Unnecessary debug messages in adintool.
    - Several bugs around reading / applying "-cmnload".
    
    4.3 (2013.12.25)
    New features:
    - FBANK and MELSPEC support.
    - Network-based feature vector and outprob vector input.
    - Static mean/variance for cepstral mean/variance normalization.
    - State output probability (i.e. outprob) vector input for DNN-HMM decoding.
    - State ID "" extension of hmmdefs for DNN-HMM decoding.
    - Real-time feature extraction and network transmittion by 'adintool'.
    
    Modified:
    - "mkbinhmm" now keeps the state order and id of the original hmmdefs.
    - For portaudio, pause / resume operation synced between engine and audio I/O
    - Load / save cepstral mean/variance of CMN/CVN in HTK text format.
    
    New options:
      [-input vecnet]       read feature / outprob vectors from network
      [-input outprob]      read outprob vectors from HTK parameter file
      [-outprobout [file]]  save computed outprob vectors to HTK file (for debug)
    
    4.2.3 (2013.6.30)
    New features:
    - Add function "j_reload_adddict()" to reload dictionaries.
    - Add option "-lvscale factor" and func "j_adin_change_input_scaling_factor()"
    to scale the amplitude of captured audio by the factor.
    - Add option "-rejectlong msec" to reject too long input.
    - Add minimum bayes risk decoding, contributed by H. Nanjo and R. Furutani
    - Support binary N-gram symbol charset conversion by "mkbingram".
    
    Fixes:
    - Fix sending audio stream via network with incorrect byte order at
    big-endian machines.
    - Fix occasional failure of closing audio device at j_close_stream().
    - Fix segfault when reading binary hmm created at 64bit env. with embedded parameters.
    - Fix memory leak when failed to read an N-gram file.
    - Fix memory leak when input length overflow is detected.
    - Fix unable to load feature vector plugin.
    - Update microphone input code for recent MacOSX.
    
    4.2.2 (2012.8.1)
    Fixes:
    - Now can be compiled without flex library
    - Fix failure of reading binary N-gram when compiled with "--enable-words-int"
    - Fix incorrect handling of file paths with backslash in jconf file at Windows
    - Fix segfault when reading an errorous word dictionary.
    - Fix occasional segfault which may occur while search.
    
    4.2.1 (2011.12.25)
    New features:
    - Add support for per-word insertion penalty setting at grammar
      recognition. You can set different word insertion score for each word
      entry at .dict file. For example, if you have an entry
            15 [a] a
      in .dict file and want to assign word insertion score of "-2.0" to
      this word, you can write like this:
            15 @-2.0 15 [a] a
      The figure after "@" is the insertion penalty. The third
      element should be the same as the first element.
    
    - New option "-chunk_size" can specify the audio fragment size in
      number of samples. The default value is 1000.
    
    - At "adintool", enable input detection by default for standard input.
    
    Fixed bugs:
    - (IMPORTANT) CMN is not performed for C0 coef.  This bug exists in
      the versions from 4.1.3 to 4.2.
    - "-forcedict" won't work for additional dictionaries given by "-adddict".
    - Corrupted header of recorded WAV file when interrupted by CTRL+C.
    - Occasional segfault when reading a wrongly formatted dictionary.
    - Won't compile with configure option "--enable-word-graph".
    - Segfault of "mkbingram" and "generate-ngram" at cygwin.
    
    4.2 (2011.5.1)
    New features:
    - Additional score-based pruning at the 1st pass.  It is disabled by
      default, you can enable by using an option "-bs arg". The argument
      is score range.
    - New support for PulseAudio (--with-mictype=pulseaudio)
    - New Option "-adddict", "-addword" to read additional dictionaries / words.
    - Portaudio library updated to V19.  Audio capture device can be
      changed by env. "PORTAUDIO_DEV_NUM".  The device list will be
      output at start up.
    
    Changed behavior:
    - "mkbinhmmlist" now saves pseudo phone list extracted from AM for
      faster start up. The output should be used with the same AM
      specified at generation.  Note that the converted binhmmlist file
      can not be used with older Julius.
    - Audio library linking was modified at configure script.
      When "--with-mictype=..." is explicitly specified, Julius will link
      ONLY the audio library.  If not specified, Julius will link all the
      audio devices whose development file was detected by the configure.
    
    Library functions:
    - j_config_load_string_new(char *str): like j_config_load_file(), but
      parse the given string to set parameters.
    - add_dict(), add_word(): the same as "-adddict" and "-addword".
      (They should be called at start up before starting engine)
    - (portaudio/Windows) j_open_stream(recog, NUMSTR) to choose device NUM.
      ex. 'j_open_stream(recog, "1")' will open device number one.
    - (portaudio/Windows) get_device_list(): obtain list of available devices.
    
    Fixes:
    - Improved tree lexicon structure for better memory management.
    - Reduce malloc calls at reading N-gram.
    - Eliminated memory leaks using Valgrind.
    - Workarounds to avoid crash with j_close_stream().
    - Now allow "-iwsp" only with multi-path acoustic model.
    
    4.1.5.1 (2010.12.25)
    Modified:
     - Fixed problem related to the license.
    
    4.1.5 (2010.6.4)
    Bug fixes:
     - Language model / decoding (these bugs may affect the ASR performance):
       - Several wrong word insertion penalty handling on grammar was found and fixed.
       - Now correctly add the prob. of the first word at the second pass.
     - MFCC computation:
       - Support MFCC computation when liftering parameter (CEPLIFTER) = 0.
     - Compilation:
       - Fixes to build Julius on cygwin and MSVC.
       - Supports "gcc -mno-cygwin" on cygwin.
       - Compilation error with configure "--disable-plugin"
     - Module mode:
       - Unable to send grammar from jcontrol.
       - Not working "DELPROCESS" command when SR and LM has different names.
     - Other fixed bugs:
       - wrong parsing of "-mapunk" option.
       - "-htkconf" in a jconf file now correctly handles the file path as relative to the jconf file.
       - "-input stdin" now supports WAV format.
       - not working "-plugin DIRNAME" on Win32/MSVC.
    
    4.1.4 (2009.12.25)
    New feature:
     - added function to choose input audio device on MSVC compiled Julius,
       by specifying a device ID with env. var. "PORTAUDIO_DEV_NUM".
       The available device IDs will be listed in the system log at start up.
     - You can now set a locale for a LM in Julius.cpp.
    
    Bug fixes:
     - now can be compield on Mac OS X (OS X 10.6 SDK).
     - fixes around portaudio for smaller latency and compatibility (Windows).
    
    4.1.3 (2009.11.2)
    New features:
     - new MSVC support: please read "msvc/00README.txt"
     - extended N-gram to support arbitrary N
     - portaudio external library (V19) can be used instead of internal V18.
       When configure detects portaudio library installed in your system,
       Julius will use it instead of internal V18.  You can also choose
       input device by "PORTAUDIO_DEV" env. var. at V19library.  See the
       log text at start up to know how to set it.
     - allow word alignment output (-walign) in module mode
    
    Modified:
     - ! now Julius do not perform CMN on 0'th cepstral coefficients,
       which is the same as the old 4.0.x versions.
     - j_get_current_filename() added on JuliusLib
     - improved "--enable-wpair" handling
    
    Bug fixes:
     - many bugs around audio open/close API on JuliusLib
     - fail to do make in julius-simple
     - unable to record inputs at cygwin
     - segfault on adintool with "-server"
     - occasional segfault at grammar recognition
    
    4.1.2 (2009.2.12)
    [SRILM support]
    - Added swapping "<s>" and "</s>" when reading BACKWARD ARPA file trained by SRILM. It will be automatically detected. If detection fails, you can specify an option "-swap" in mkbingram to do that.
    - Internally modify the unigram probability of "<s>" or "</s>", since they may be set to "-99" in SRILM model. The same value as opposite will be assigned.
    [N-gram]
    - Size limit extended from 2GB to 4GB for big N-gram.
    - "<unk>" and "<UNK>" can be changed by "-mapunk".
    - More strict check for unknown words: Julius now terminates with error when dictionary has OOV words and N-gram is not open (no unk word).
    [Improvements]
    - Faster successor list building algorithm
    - Update yomi2voca.pl to cover more minor Japanese pronunciation.
    - Workaround for audio buffer overrun in ALSA
    [JuliusLib]
    - Added API function "j_close_stream()" to exit main recognition loop.
    [Bug Fixes]
    - Fixed segfault on adintool when specifying multiple servers.
    - Fixed compilation error on cygwin (libesd)
    - Fixed segfault when not specifying "-input" option.
    
    4.1.1 (2008.12.13)
    Bug fixes:
    
    [N-gram]
    - sometimes could not read an ARPA N-gram file trained by SRILM.
    [A/D-in]
    - "-input stdin" does not work.
    - "SOURCERATE" at "-htkconf" is ignored.
    [Forced alignments]
    - now can be used in isolated word recognition and with "-1pass".
    - "-palign", "-walign" and "-salign" can not be run together at a time.
    [Module mode]
    - freezes when a grammar is specified by its ID number.
    - wrong grammar ID in recognition result (GRAM=.. always 0)
    - "SYNCGRAM" will cause crash at isolated word recognition.
    - unable to receive/activate/dactivate on isolated word recognition.
    [Others]
    - fails to compile on several OS (needs "-ldl").
    - does not handle backslash escaping correctly in Jconf file.
    - does not output the 1st pass result as a final result with "-1pass".
    [Tools]
    Jcontrol
    - does not support "graminfo" command.
    - can not send a dictionary to Julius running isolated word recognition.
    mkdfa
    - segfault on mkfa
    - fails to read a grammar file on DOS format.
    adintool
    - wrong behavior when splitting a long audio file.
    - now output time of each segment.
    
    4.1 (2008.10.03)
    New plugin extension:
      - supported types:
          - A/D-in plugin
          - feature vector input plugin
          - audio input monitor / postprocess plugin
          - feature vector monitor / postprocess plugin
          - result plugin
          - can add arbitrary JuliusLib callback via plugin
      - sample codes is included, with full documentation of function spec.
      - run on Linux, Windows and other unix variants with dlopen() capability
    
    Newly supported features:
      - multi-stream feature input
      - MSD-HMM (compatible with "HTS" toolkit)
      - CVN 
      - frequency warping for VTLN (no estimation yet)
      - "-input alsa", "-input oss" and "-input esd"
      - perl version of jcontrol client "jclient-perl"
    
    Modified:
      - Restrict option orders when multiple instances defined (-AM, -LM, -SR):
          - Option should be just after correspondence instance declaration. 
            (ex. LM options should be placed after "-LM" and before other 
            instance declaration.)
          - Global option should be before any instance declaration, or
            just after "-GLOBAL" option.
        This new restriction can be removed by "-nosectioncheck" option.
    
    Fixed bugs:
      - "-record" fails to record the first silence part!
      - Not working "-multigramout"
      - environment variable expansion sometimes fail within jconf file.
      - limits extended:
         maximum HMM name length = 256 char, Number of HMM states unlimited.
      - Module mode error message on grammar command.
    
    Documents:
      - Alpha version of "Juliusbook" (contains only manuals at this time)
      - Unix manuals are moved to "man" directory.
    
    4.0.2 (2008.05.27)
    New features:
    
      - New option "-fallback1pass" will output 1st pass result as final result
        when the 2nd pass fails.
      - Added support for "USEPOWER=T" on feature extraction.
    
    Modified:
      - "-AM_GMM" becomes optional: GMM will share AM params if not specified.
    
    Fixed:
      - GMM rejection does not work (since 4.0.1)
      - Cannot specify other A/D device on Linux/ALSA correctly.
      - Sometimes fails to read a big N-gram.
      - Sometimes crush with "-record" option.
      - Callback timing modified on real-time input with sp-segment/GMM/VAD.
      - Other minor fixes.
    
    4.0 (2007.12.19)
    - Re-constructed all data structures and re-organize source code.
    - Core engine now becomes a library called JuliusLib, with API and callbacks.
    - Multi-model decoding now available.
    - Modularize language model handling, and merge Julian to JuliusLib.
    - Support longer N-gram (N > 3).
    - User-defined LM function support.
    - Handy isolated word recognition mode.
    - Confusion network output.
    - Improvements in short-pause segmentation, especially for live input.
    - GMM-based VAD.
    - Decoder-based VAD.
    - Integrated many compile-time options.
    - Reduce memory usage.
    - Sample application to use the JuliusLib is included: "julius-simple".
    - Update tools:
       - "adintool" supports multi-server mode.
       - "generate-ngram" newly added to generate sentences from N-gram
    
    3.5.3 (2006.12.29)
      o  Improved Performance:
         - acoustic computation optimized: now becomes 20%-40% faster!
         - optimize memory access: re-use work area of deleted hypothesis
           in the 2nd pass.
         - some memory allocation improvement on dictionary and word trellis.
    
      o  New Grammar Tools:
         - "dfa_minimize", "dfa_determinize" will minimize/determinize DFA.
            mkdfa.pl now calls dfa_mimize in it.
         - "slf2dfa": a toolkit to convert HTK slf to Julian dfa (separate kit)
    
      o  Embedding HTK Acoustic Parameters:
         - add option to load HTK Config file to set correct acoustic parameter
           configuration at recognition time. 
         - the acoustic parameter configuration can be embedded into
           header of a binary HMM file.
    
      o  Improved Word Graph:
         - add an option to completely separate graph words: words with
           different phone contexts can be output separatedly by
           "-graphrange -1".
    
      o  Support for online energy normalization:
         - Preliminary support for live recognition using acoustic model with
           energy normalization. (approximate with maximum energy of last input)
    
      o  Code refinements:
         - re-organize libsent/src/wav2mfcc.
         - modularize acoustic parameter (Value) handling.
         - output compile-time configuration of libsent with "--setting" option.
         - Doxygen 1.5.0 support.
         - "julius-info@lists.sourceforge.jp" becomes the official contact address.
         - fixed typo on copyright notice.
    
      o  Fixed bugs:
         - sometimes unable to read a binary LM on "--enable-words-int".
         - memory leaks around option handling, global variables and local buffers.
         - segmentation fault on very long input.
         - doublely counted initial state of DFA.
         - mkdfa.pl: unable to find mkfa on some OS.
         - adintool: makes empty output file on termination.
         - adintool: miss last inputs when killed.
         - other small changes.
    
    3.5.2 (2006.07.31)
      o  Speed-up and improvement on Windows console:
         - Support DirectSound for better input handling
         - Support input threading utilizing callback API on portaudio.
         - Support newest MinGW (tested on 5.0.2)
    
      o  More accurate word graph output:
         - Add option to cut the resulting graph by its depth
           (option -graphcut, and enabled by default!)
         - Set limit for post-processing loop to avoid infinite loop
           (option -graphboundloop, and set by default)
         - Refine graph generation algorithm concerning dynamic word merging
           and search termination on the second pass.
    
      o  Add capability to output word graph instead of trellis on 1st pass:
         - 1st pass generates word graph instead of word trellis as
           intermediate result by specifying "--enable-word-graph".
           In that case, the 2nd pass will be restricted on the graph, not
           on the whole trellis.
         - With "--enable-word-graph" and "--enable-wpair" option, the
           first pass of Julius can perform 1-pass graph generation based
           on 2-gram with basically the same algorithm as other popular
           word graph based decoders.
    
      o  Bug fixes:
         - configure script did not work on Solaris 8/9
         - "-gprune none" did not work on tied-mixture AM
         - Incorrect error message for AM with duration header other than "NULLD"
         - Always warns about zero frame stripping upon MFCC
    
      o  Imprementation improvements:
         - bmalloc2-based AM memory management
    
    3.5.1 (2006.03.31)
      o  Wider MFCC types support:
         - Added extraction of acceleration coefficients (_A).  Now you
           can recognize waveform or microphone input with AM trained with _A.
         - Support all MFCC qualifiers (_0, _E, _N, _D, _A, _N, _Z) and their
           combination 
         - Support for any vector lenth (will be guessed from AM header)
         - New option: "-accwin"
         - New option "-zmeanframe": frame-wise DC offset removal, like HTK
         - New options to specify detailed analysis parameters (see manual):
              -preemph, -fbank, -ceplif, -rawe / -norawe, 
              -enormal / -noenormal, -escale, -silfloor
    
      o Improved microphone / network recognition by MAP-CMN:
         - New option "-cmnmapweight" to change MAP weight
         - Option "-cmnload" can be used to specify the initial cepstral
           mean at startup
         - Cepstral mean of last 5 second input is used as an initial mean
           for each input.  You can inhibit updating of the initial mean
           and keep the value loaded by "-cmnload" by option "-cmnnoupdate".
    
      o Module issue:
         - Julius now outputs "" when recognition starts, and
           "" after recognition stopped by module command.
           Use this for safer server-client synchronization.
         - now can specify grammar name from client by specifying a name
           after a command like "ADDGRAM name" or "CHANGEGRAM name".
    
      o Bug fixes:
         - Sometimes segfault on pause/resume command on module mode while input.
         - Can not read N-gram with tuples > 2^24.
         - Can not read HMM with 3-state (1 output state) model on multi-path.
         - Sometimes omit the last transition definition in DFA file.
         - Sometimes fails to compile the gramtools on MacOSX.
    
    3.5 (2005.11.11)
      o  New features:
         - Input verification / rejection using GMM (-gmm, -gmmnum, -gmmreject)
         - Word graph output (--enable-graphout, --enable-graphout-nbest)
         - Pruning on 2nd pass based on local posterior CM (--enable-cmthres)
         - Multiple/per-grammar recognition (-gram, -gramlist, -multigramout) 
         - Can specify multiple grammars at startup: "-gram prefix1,prefix2,..."
           or "-gramlist listfile" where listfile contains list of prefixes.
         - General output character set conversion "-charconv from to"
           based on iconv (Linux) or Win32API+libjcode (Windows)
    
      o  Improved audio inputs on Linux:
         - ALSA-1.x support. (--with-mictype=alsa)
         - EsounD daemon input support. (--with-mictype=esd)
         - Fixed some bugs on USB audio input.
         - Audio capturing device can be specified via env. "AUDIODEV".
         - Extra microphone API support using portaudio and spLib API.
    
      o  Performance improvements:
         - Reduced memory size for beam operation on the 1st pass.
         - Slightly optimized tree lexicon by removing redundant data.
         - Reduced size of word N-gram index (reduced from 32 bit to 24 bit).
    
      o  Fixed bugs:
         - Not working spectral subtraction.
         - Memory leak when stack exhausted ("stack empty") on 2nd pass.
         - Segmentation fault on a very short input of 1 to 4 frames.
         - AM trained with no CMN cannot be used with waveform/mic input.
         - Wrong short-pause word handling on successive decoding mode.
           (--enable-sp-segment)
         - No output of "maxcodebooksize" at startup.
         - No output of the number of sentences found when stack exhausted.
         - No output of "-separatescore" on module mode.
         - Beam width does not adjusted when grammar has been changed and 
           full beam options (-b 0) is specified in Julian.
         - Wrong update of category-aware cross-word triphones when
           dynamically switching grammar on Julian.
         - No output of grammar to stdout on multiple grammar mode.
         - Unable to send/receive audio data between different endian machines.
         - (Linux) crash when compiled with icc.
         - (Linux) some strange behavior on USB audio.
         - (Windows) confuse with CR/LF newline inputs in several text inputs.
         - (Windows) mkdfa.pl could not work on cygwin.
         - (Windows) sometimes fails to read a file when not using zlib.
         - (Windows) wrong file suffix when recording with "-record" (.raw->.wav)
    
      o  Unified source code:
         - Linux and Windows version are integrated into one source.
         - Multi-path version has been integrated with the normal version
           into one source.  The multi-path version of Julius/Julian, that
           allows any transitions of HMMs including model skip transition,
           can be compiled by "--enable-multipath" option.  The part of
           source codes for the multi-path version can be identified 
           by the definition "MULTIPATH_VERSION".
    
      o  Other improvements:
         - Now can be compiled on MinGW/MSYS on Windows
         - Totally rewritten comments in entire source in Doxygen format.
           You can generate fully browsable source documents in English.
           Try "make doxygen" at the top directory (you need doxygen installed)
         - Install additional executables of julius/julian with version and setting
           names like "julius-3.5-fast" when "make install" is invoked.
         - Updated LICENSE.txt with English translation for reference.
    
      o  Changed behaviors:
         - Binary N-gram file format has been changed for smaller size.
           The old files can still be read directly by julius, in which
           case on-line conversion will be performed at startup.
           You can convert the old files (3.4.2 and earlier) to the new
           format with the new mkbingram by involing the command below:
    	       "mkbingram -d oldbinary newbinary"
           Please note that since mkbingram now output the new format
           file, it can not be read by older Julius.
           The binary N-gram file version can be detected by the first 17
           bytes of the file: old format should be "julius_bingram_v3" and
           new format should be "julius_bingram_v4".
         - Byte order of audio stream via tcpip fixed to LITTLE ENDIAN.
         - Now use built-in zlib by default for compressed files.  This may
           make the engine startup slower, and if you prefer, you can still
           use the previous method using external gzip command by specifying
           "--disable-zlib".
         - (Windows) Changed the compilation procedure on VC++.  You can build
           Julian by only specifying "-DBUILD_JULIAN" at compiler option,
           and do not need to alter "julius.h".
    
    3.4.2 (2004.05.07)
    - New option "-rejectshort msec" to reject short input.
    - More stable PAUSE/RESUME on module mode with adinnet input.
    - Bug fixes:
      - Memory leak on very short input.
      - Missing Nth result when small vocabulary is used.
      - Hang up of "generate" on small grammar.
    - Cosmetic changes:
      - Cleanup codes to confirm for 'gcc -Wall'.
      - Update of config.guess and config.sub.
      - Update of copyright to 2004.
    
    3.4.1 (2004.02.25)
    - Search algorithm is slightly modified to make search more stable at
      of 2nd pass.  These modification are enabled by default, and
      MAY IMPROVE THE RECOGNITION ACCURACY as compared with older versions.
      - fixed overcounting of LM score for the expanded word.
      - new inter-word triphone approximation (-iwcd1 best #) on 1st
        pass.  This new algorithm now becomes default.
    - Newly supports binary HMM (original format, not compatible with HTK).
      A tool "mkbinhmm" converts a hmmdefs(ascii) file to the binary format.
    - MFCC computation becomes faster by sin/cos table lookup.
    - Bugs below have been fixed:
      - (-input adinnet) recognition does not start immediately after speech
    		     inputs begin when using adinnet client.
      - (-input adinnet) together with module mode, speech input cannot
    		     stop by pause/terminate command.
      - (-input adinnet) unneccesary fork when connecting with adinnet client.
      - (-input rawfile) error in reading wave files created by Windows
                         sound recorder.
      - (CMN) CMN was applied any time even when acoustic models does not want.
      - (AM) numerous messages in case of missing triphone errors at startup.
      - (adintool) immediately exit after single file input.
      - (sp-segment) fixed many bugs relating short pause word and LM
      - (sp-segment) wow it works with microphone input.
      - (-[wps]align) memory leak on continuous input.
    - Add option to remove DC offset from speech input (option -zmean).
    - (-module) new output message:
      '<INPUTPARAM FRAMES="input_frame_length" MSEC="length_in_msec">'
    - Optional feature "Search Space Visualization" is added (--enable-visualize)
    - HTML documentations greatly revised in doc.
    
    New argument: "-iwcd1 best #" "-zmean"
    New configure option: "--disable-lmfix", "--enable-visualize"
    
    3.4 (2003.10.01)
    - Confidence measure support
      - New parameter "-cmalpha" as smoothing coef.
      - New command "-outcode C" to output CM in module output
      - Can be disabled by configure option "--disbale-cm"
      - Can use an alternate CM algorithm by configure option "--enable-cm-nbest"
    - Class N-gram support
      - Can be disabled by configure option "--disable-class-ngram"
      - Factoring basis changed from N-gram entry to dictionary word
    - WAV format recording in "adinrec", "adintool" and "-record" option
    - Modified output message
        startup messages,
        engine configuration message in --version and --help,
    - Fixes:
        some outputs in module mode,
        bug in only several frame input (realtime-1stpass.c),
        long silence at end of segmented speech
        miscompilation with NetAudio,
        word size check in binary N-gram,
        bug in acoustic computation (gprune_none.c).
        "-version" -> "-setting", "-hipass" -> "-hifreq", "-lopass" -> "-lofreq"
    
    3.3p4 (2003.05.06)
    - Fixes around audio input:
      - Fix segfault/hangup when microphone input runs for a long period.
      - Fix client hangup when input speech is too long in module mode.
        (now send an buffer overflow message to the client instead of hangup)
      - Fix audio buffering for very short input (<1000 samples).
      - Fix blocking in tcpip adin.
    - Some cosmetic changes (jcontrol, LOG_TEN, etc.)
    
    3.3p3 (2003.01.10)
    - New inter-word short pause handling:
      - [Julius] New option added for short pause handling.  Specifying
        "-iwspword" adds a short-pause word entry, namely " [sp] sp sp",
        to the dictionary.  The entry content to be changed by using "-iwspentry".
      - [multi-path] Supports inter-word context-free short pause handling.
        "-iwsp" option automatically appends a skippable short pause model at
        every word end.  The added model will also be ignored in context
        modeling.  The short pause model to be appended by "-iwsp" can be
        specified by "-spmodel" options.  See documents for details.
    - Fixes for audio input:
       - Input delay improved: the initial response to mic input now
         becomes much faster than previous versions (200ms -> 50ms approx.).
       - Would not block when other process is using the audio device, but
         just output error and exit.
       - Update support for libsndfile-1.0.x.
       - Update support for ALSA-0.9.x 
         (to use this, add "--with-mictype=alsa" to configure option.)
    
    patch for libsndfile-1.0.x (2002.11.19)
    - This patch fixes compilation error with libsndfile > 1.0.x.
    
    3.3p2 (2002.11.18)
    - Newly supports model-skip transition.  From this version, you can
      use "any" type of state transition in HTK format for acoustic model.
      (see Bugs above for limitation).
    - add new feature: "-record dir" records speech inputs sucessively
      into the specified directory.
    - fix segfault on Solaris with "-input mfcfile".
    - fix adin-cut bug when using module mode and adinnet together.
    - fix output flush after last recognition output.
    
    3.3p1 (2002.10.15)
    Fixed the following bugs:
    - Fixed incorrect default value of language weights for second pass (-lmp2).
    - Fixed sometimes read failure of dictionary file.
    - Fixed wrong output of "-separatescore" together with monophone
    model.
    
    
    3.3 (2002.09.12)
    The updates and new features from rev.3.2 is as shown below.
    
    - New features added:
    	- Server module mode - control Julius (input on/off, grammar switching)
    	  from other client process via network.
    	- Online grammar changing and multi-grammar recognition supported.
    - Noise robustness:
    	- Spectral subtraction incorporated.
    - Support more variety of acoustic models:
    	- "multi-path version" is available that allows any transition
    	   including loop, skip and parallel transition.
    - A little improvement of recognition performance by bug fixes
    - Other minor extensions (CMN parameter saving, etc.)
    - Many bug fixes
    
    English documents are available in
      o online manuals (will be installed by default), and
      o Translated full documentation in PDF format: Julius-3.2-book-e.pdf.
    We are sorry that current release contains only documents for old rev.3.2.
    We are now working to update it to catch up with the current rev.3.3 version.
    

浅谈开放词汇目标检测 在过去的研究中,我们看到了开放词汇物体检测领域的各种创新和进步。这些研究涵盖了从基于AI的物体检测翻译应用到理解机器人中的物体描述,再到使用视觉-语言匹配进行开放词汇物体检测等多个方面。此外,一些研究还探讨了如何通过改进伪标签、利用细粒度描述符以及探索多模态上下文知识等方法来提高开放词汇物体检测的效果。然而,尽管取得了显著的进步,但开放词汇物体检测领域仍存在许多挑战和问题需要解决。 阅读详情

相关推荐

Open-Vocabulary Semantic Segmentation via Attribute Decomposition-Aggregation学习笔记

开放词汇语义分割,需要在推理时分割出新的类别。最近研究探索了视觉语言预训练来处理,但在实际场景中受到低质量的文本类别名称的影响。 例如,当遇到简短或不完整的名称产生歧义、未出现在预训练的词典中的新词以及用户难以描述的类别时,通常会出现例外情况。 为了解决这些问题,这项工作提出了一个新的分解聚合框架,灵感来自于人类理解新概念的认知。 具体来说,在分解阶段,将类名解耦为不同的属性描述,以丰富语义上下文。在聚合阶段,将不同的属性组合成一个完整的全局描述,形成一个区分目标对象和其他对象的判别分类器。

Bai__Yang_的博客 2576

【开放域目标检测】一:Open-Vocabulary Object Detection Using Captions论文讲解

【开放域目标检测】一:Open-Vocabulary Object Detection Using Captions论文讲解

wzk4869的博客 5200

Open-Vocabulary 论文速览201-300(arxiv 截止2024.06.12)

Open-Vocabulary Detection && Segment &&3D论文内容概览

weixin_44287798的博客 3741

开放词汇含义

我们提出的解决这个问题的方法是联合图像像素与词概念嵌入框架;然而,现实世界中的物体和场景是多样的,难以用一个固定的标签集合来描。为了应对这一挑战,研究者们提出了开放词汇(Open Vocabulary)的概念。景 时,能够 自我更新 并学习到新的标签。在传统的计算机视觉任务中,通常会有一个固定的标签集合,即封闭词汇(Closed。指的是一种 可扩展的标签集合,它允许计算机视觉系统在遇到 新的物体或场。Open- Vocabulary 是在哪篇文章被率先提出。

danger2的博客 1176

ViLD(Open-Vocabulary Object Detection via Vision and Language Konwledge Distillation)

基于视觉语言(图像文本)知识蒸馏的目标检测。 CLIP模型当成teacher模型,去蒸馏自己的模型,从而达到zero-shot。

weixin_45104951的博客 6073

HTK3.4程序员手册(2.3)--特征参数提取HParm.c

HTK3.4程序员手册(2.3)--特征参数提取HParm.c by 云龙 HTK book中提到的参数有11种: "LPC", "LPREFC", "LPCEPSTRA",   "LPDELCEP", "IREFC", "MFCC", "FBANK", "MELSPEC","DISCRETE", "PLP","ANON" 但HTK3.4中是否都支持呢? 请看Conver

云龙 2188

【多模态】24、开放词汇学习到底是什么?

本文主要介绍开放词汇学习

呆呆的猫的博客 8711

Open-vocabulary-entity-type-description:生成细粒度的开放词汇实体类型描述

项目名称 该项目是ACL 2018论文的PyTorch实施:生成细粒度的开放词汇实体类型描述 先决条件 Python 2.7+ PyTorch 跑步 python <model>.py 这将完成模型的训练,验证和测试。 测试数据的结果将被写入名为“ result_ <模型名称> .csv”的文件。 例如,要运行我们的模型,请使用 python model.py 评估 要运行评估脚本,请使用 python evaluate.py result_<model>.csv 作者 Rajarshi Bhowmik- Gerard de Melo- 引文 如果您使用此代码,请引用我们的论文。 @inproceedings{Bhowmik2018EntityDescriptions, title = {Generating Fine-Grained Open Vocabular

Open-Vocabulary Multi-Label Classification via Multi-modal Knowledge Transfer 论文解读

Open-Vocabulary Multi-Label Classification via Multi-modal Knowledge Transfer 论文解读

weixin_46112284的博客 2287

Matlab实战:3种雷达间歇采样干扰的仿真与识别(附完整代码)

本文通过Matlab实战,详细仿真了雷达有源间歇采样干扰的三种核心样式:直接转发、重复转发和循环转发干扰。文章提供了完整的代码实现,并深入探讨了基于时频分析和循环平稳分析的特征提取与识别方法,为雷达抗干扰技术的学习与实践提供了清晰的工程实现路径。

weixin_29053695的博客 364

Zynq7020实战:ARM Cortex-A9降频避坑指南(附温度对比测试)

本文详细介绍了如何在Zynq7020平台上对ARM Cortex-A9处理器进行降频优化,以降低功耗并提升系统稳定性。通过分析时钟架构、提供实战操作步骤和温度对比测试数据,帮助开发者有效避免降频过程中的常见问题,实现嵌入式系统的高效能耗管理。

root9的博客 206

Open-Vocabulary论文内容速览(301-379 arxiv截止2024.06.12)

Open-Vocabulary Detection && Segment&&3D 论文内容概览

weixin_44287798的博客 2082

lammps 案例in文件汇总

进阶案例: 聚乙烯和石墨烯界面pull out模拟代码 反应力场reax/c设置方法及案例解析 石墨烯热导率模拟代码(NEMD方法) 圆锥形SiC纳米磨削单晶硅源代码 高熵合金FeNiCrCoCu纳米压痕模拟代码 fix indent命令实现纳米摩擦及摩擦力计算 Fe原子辐照轰击多层石墨烯模拟代码 铝基石墨烯复合物压缩变形in文件源代码 石墨烯自动卷曲到Fe纳米线模拟源代码 铝/石墨烯复合物结晶过程lammps模

lammps_jiayou的博客 9446

OPEN-VOCABULARY OBJECT DETECTION VIAVISION AND LANGUAGE KNOWLEDGE DISTILLATION

通过视觉和语言知识提炼进行开放词汇的物体检测摘要引言 考虑到图1,我们是否可以设计出超越只识别训练标签中存在的基本类别(如玩具)的物体检测器,并扩大词汇量以检测新的类别(如玩具大象)?在本文中,我们旨在训练一个开放词汇的物体检测器,只使用基础类别的检测注释,并检测文本输入所描述的任何新类别的物体。现有的物体检测算法通常只学习检测检测数据集中的类别。增加检测词汇量的一个常见方法是收集具有更多标记类别的图像。研究界最近收集了具有大量词汇的新物体检测数据集(Gupta等人,2019;Kuznetsova等人,20

pingguolou的博客 3397

15、目标检测模型的评估指标与YOLO算法解析

本文详细解析了目标检测模型的评估指标,包括精确率、召回率、AP、mAP及IoU的定义与作用,并深入介绍了YOLO算法的核心原理与推理流程。从网格划分、锚框引入到边界框细化与后处理,全面阐述了YOLO如何实现快速准确的目标检测。同时总结了YOLO在智能安防、自动驾驶和工业检测等领域的应用,并展望了其未来在准确性提升、多类别支持和技术融合方面的发展趋势。

work3的博客 65

【LAMMPS学习】四、运行 LAMMPS(2)命令行选项

本部分解释了在安装可执行文件或下载源代码并构建可执行文件后如何运行 LAMMPS。命令文档页面描述了输入脚本的结构以及它们可以包含的命令。

荷塘阅色 5242

Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Models

联合CLIP和Diffusion的开集语义分割https://img-blog.csdnimg.cn/77b4e88f96c742109e5796053affffb1.png框架

孤独的猫 2220

金融领域强化学习在高频交易中的应用

强化学习(Reinforcement Learning,简称RL)是一种机器学习范式,通过智能体与环境的交互来学习策略,以最大化累计奖励。与监督学习和无监督学习不同,强化学习依赖于动态的反馈机制,智能体通过试错不断优化决策过程。高频交易(High-Frequency Trading,简称HFT)是指在极短时间内进行的大量金融交易,通常利用算法和计算机系统自动执行交易策略。高频交易依赖于高速的数据处理和决策能力,以捕捉市场中的微小价格波动。状态(State):环境在某一时刻的观测,反映了系统的当前情况。

AI天才研究院 893

Praat脚本-030 | 谈谈自动标注这点儿事(3)

目录简介安装环境安装Python安装WxPython安装JuliusSPPAS使用数据格式数据准备准备音频和文本导入SPPAS进行切分关注版权说明 简介 SPPAS工具是由Brigitte Bigi(Laboratoire Parole et Langage - Aix-en-Provence - France)开发的。在网上提供免费下载及使用。这是一款目前比较流行的自动标注,音段对齐的工具,目前应用于法语、英语、汉语等都有不错的效果。虽然音段自动对齐的准确率并不完善,但是对于一般的语音研究者来说,能够自动

极地语音工作室:专注语音处理、Praat脚本、TextGrid标注、声学分析与相关AI工具实践。 1917

uniapp - 解决uni.getLocation定位不准有偏差,定位经纬度不准确有几百米误差、获取地图定位有偏差怎么办,已配置高精确度但依旧不精准(uni.getLocation定位不准如何解决)

uni.getLocation,获取当前的地理位置、速度。已经设置altitude高精度,uniapp API获取定位有差异问题解决,获取定位有几百米偏差,uni.getLocation(OBJECT)中使用默认type ”wgs84“会导致定位不准的问题,gcj02,uni.getLocation地图定位不准,获取经纬度错误偏移了几十公里,经纬度的精确度误差,怎么消除经纬度偏差,解决 uniapp uni.getLocation 定位经纬度不准问题,使用定位不准,定位不准误差问题,解决定位不准问题,高德地

✅ 精选推荐文章 1万+

【部署优化篇三】《DeepSeek边缘计算实战:把目标检测模型塞进树莓派,让AI在巴掌大的设备上“开天眼“》

谁说只有超级计算机才能跑AI?今天咱们就要在树莓派上玩转DeepSeek目标检测,让这个巴掌大的小盒子变成会‘看’世界的智能终端!本文手把手教你从零开始,把最潮的目标检测模型塞进树莓派。全程高能预警,建议准备好你的树莓派4B/5和散热风扇,咱们这就开启边缘计算的魔法之旅!

商务合作|问题讨论|交流学习 请联系作者微信,加微信请务必注明来意,博客主页有联系方式 1306

Open-vocabulary 论文速览1-100篇(arxiv截止时间2024.6.12)

Open-Vocabulary Detection && Segment && 3D && other

weixin_44287798的博客 2532

NTscan扫描工具

ntscan是一个扫描器,可以扫描没有密码和弱密码的共享,先到配置文件中填写猜测的用户名和密码,扫描过程中右侧出现的是扫描日志。

Open-Vocabulary SAM本地部署运行

Open-Vocabulary SAM,简称ovsam,项目对应论文:Open-Vocabulary SAM: Segment and Recognize Twenty-thousand Classes Interactively。该项目提供了在线使用的demo,也可以下载后本地运行。本文介绍ovsam本地部署运行。

Duang2001的博客 2118

文档:亲测可用解决vnc灰屏,黑屏,鼠标大黑叉情况

主要解决vnc-viewer灰屏,黑屏,鼠标大黑叉等现象,详细见文档。

大中型超市收银系统(可直接运行)

光步前台收银系统系统功能:商品销售、商品退货、挂单、收银员权限、会员销售、单品取消、整单取消、商品查询、脱机销售、脱机数据上传,基本资料下载,开钱箱、小票打印、收款结算,支持多币种付款方式(现金、储值卡、礼券、微信等),支持多种促销。运行环境:.net 4.0登陆信息用户名:1001 密码:123456内置商品信息,商品编码可输入00001000 - 00010000,测试效果相关功能键F5 结算 F9挂单 F11商品查询 F8退货 Insert单品取消 Del整单取消 F12开钱箱F2 会员 Ctrl+Z退出系统 Home 锁屏 相关源码待公开

Open-Vocabulary Object Detection Using Captions论文讲解

Open-Vocabulary Object Detection Using Captions论文讲解

wzk4869的博客 2180

efm32 入门.pdf

本PDF共计22页,介绍的是操作驱动API,下载与否自己斟酌。

Small-Footprint Open-Vocabulary Keyword Spotting with Quantized LSTM Networks

原创: Lebhoryi@rt-thread.com 时间: 2020/04/11 文章目录0x00 Paper0x01 为了解决什么问题0x02 提出了哪些创新点0x03 文章body3.1 第二节 ACOUSTIC MODEL3.2 第三节 KEYWORD SPOTTING METHOD3.3 第四节 EXPERIMENTAL SETUP0x04 other 0x00 Paper p...

lebhoryi 974

语音标注自动音段对齐工具SPPAS使用笔记

[更新日期:2019年08月08日 。咨询邮件feipengshao@163.com,请务必附件样例以及有问题脚本,出错提示等,否则不予回复] SPPAS工具是由Brigitte Bigi(Laboratoire Parole et Langage - Aix-en-Provence - France)开发的。在网上提供免费下载及使用。这是一款目前比较流行的自动标注,音段对齐的工具...

极地语音工作室:专注语音处理、Praat脚本、TextGrid标注、声学分析与相关AI工具实践。 2万+
上一篇: audio search
下一篇: audio to text
凌风探梅
博客等级 码龄19年 2575粉丝 113原创
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值