Skip to main content

A2.Mirana Utility

trydoforOriginalMiranaUtilityAbout 8 min

A2.Mirana Utility

Mirana provides useful utilities.

A2.1.anti/ Anti-engineering

  • BeanVisitor - Visist the fields of a bean via reflection, mainly for property formatting
  • G - Anti-pattern, passing values across layers
  • L - Anti-pattern, gathering message across layers

A2.2.best/ High Qulity Code

  • AssertArgs - Pre-assertion, IllegalArgumentException, BadArgsException
  • AssertMessage - Assertion, MessageException
  • AssertState Mid/Post-assertion - IllegalStateException, BadStateException
  • DummyBlock - Turn off some warns of safe code blocks
  • Param - Annotate whether param is modified
  • TypedKey - Value registration of a Map with type
  • TypedReg - Key and Value registration of a Map with type
  • Aes - Aes128 and Aes256 utility
  • Aes128 - jdk AES/CBC/PKCS5Padding, PKCS7Padding with bouncycastle
  • Aes256 - jdk AES/CBC/PKCS5Padding, Recommended for java1.8.0_162+
  • Base64 - RFC4648_URLSAFE and UTF8 by default, +/ and -_ are supported
  • Bytes - Hex and Unicode, eg. 我(25105)=>'\u6211'
  • HmacHelp - MessageAuthenticationCode HmacMD5, HmacSHA1, HmacSHA256
  • Md5 - md5sum, md5check
  • HdHelp - MessageDigest MD5, SHA1, SHA256

A2.4.cast/ Type Cast/Convert

  • BiConvertor - bi-direction converter
  • BoxedCastUtil - Conversion of wrapper and primitive
  • BoxedTypeUtil - Wrapper compatible instanceOf, isAssignable
  • EnumConvertor - enum serialization by name or qualified path
  • MethodConvertor - compact serialization of Method
  • StringCastUtil - Conversion of strings and other types
  • TypedCastUtil - Conversion of type parameters and generic types

A2.5.code/ Encode and Decode

5a.Crc4Int - int32 with CRC

Based on the original int value, generate a pseudo-random verifiable number that can decrypt the original value.

try to improve human readability and pseudo-randomness.

5b.Crc8Long, Crc8LongUtil - int64 with CRC

Based on the original long value, generate a pseudo-random verifiable number that can decrypt the original value.

The bit layout can be customized, there are 3 types by default, see Crc8LongUtil.

The applicable scenario is exposed ID number without high security and be efficiently parsed and verified in both directions.

5c.Excel26Az - Excel Column Naming

Two-way parsing by Excel column naming (A:1,B:2,... ,Z:26,AA:27)

5d.LeapCode - Pseudo-random Highly Readable Code

A textual 32-symbol notation for expressing numbers, consisting of 22 letters (minus UOIL for confusion) and 10 numbers. The encoded string looks more random, can be decrypted to its original value, and can be filled to a fixed length.

see https://www.crockford.com/base32.html

The applicable scenario is exposed ID number without high security and be efficiently parsed and verified in both directions.

5e.Other Codes

  • Mod10Code - Checking algorithm from USPS
  • RandCode - Some random numbers with better human readability
  • SlotCode - Random assignment of fixed positions, such as pickup number

A2.6. cond/ Condition assertion

  • EqualsUtil - Equivals Condition
  • StaticFlag - Global Flag
  • IfSetter - Conditional setter util

A2.7.data/ Data Transfer

  • Arr - Some Array manipulations
  • CodeEnum - Biz enum, such as I18n, status
  • DataResult - Common DTO with biz data
  • Diff - diff 2 collections, such as insertion, update, deletion
  • Null - null-safe is our goal
  • Q - Single-parameter Query Class
  • R - Scenario classes for DataResult
  • Rank - Sort by multi-conditional order
  • U - Tuple, Either for internal data transfer
  • Z - The first conditional data (such as non-null)

A2.8.dync/ Dynamic Compliation

  • Java - Dynamic compilation and creation of java
  • OrderedSpi - order and get 1st SPI

A2.9.evil Use with Caution

  • Attention - Tagging method should follow some pattern to prevent misuse
  • StartStageAttention - Methods called during the startup, do not call frequently
  • ThreadLocalAttention - Note the ThreadLocal mode
  • ThreadLocalProvider - OrderedSpi to provide ThreadLocal
  • ThreadLocalSoft - ThreadLocal using SoftReference
  • TweakingContext - Contextual configuration for debugging

A2.A.fake/ Fake Data

  • FakeDate - Generate a pseudo-random date around the given offset, guarante idempotent
  • FakeName - Generate a random chines name

A2.B.flow/ Flow Control

In high level architecture design, high-order function calls, stream processing, usually use exceptions to break in the process. Similar to the spring security system, scala's break syntax, kotlin's return@.

The following are some low-cost no-stack exceptions that control the flow, which is an anti-pattern and not recommended if not necessary.

  • FlowBreak - Static util
  • FlowBreakException - Break process with exceptions of the Enum
  • FlowEnum - flow control enum
  • FlowReturnException - Break process with return value
  • ReturnOrException - Either an exception with a return or an exception only

A2.C.func/ Function and Lambda

  • Clz - Class Util
  • Dcl - DCL of Runnable
  • Fn - Distinct and duplicate
  • Lam - Get object and method references via lambda

A2.D.i18n/ Multiple Languages

  • I18nAware - Marking interface
  • I18nEnum - Provide i18n capability for common enum
  • I18nString - String with i18n support
  • LocaleResolver - Support - and _
  • ZoneIdResolver - Case-insensitive, partial name support

A2.E.id/ Identity

e1.LightId - Lightweight Distributed ID

Lightweight distributed primary key with double buffering, using sequence to efficiently generate 64bit digital ID. ID can guarantee strict monotonic incremental (ascending order), but not continuous, and its long type 64bit composition is as follows.

  • long = 1-bit:0 fixed+8-bit: Crc+1-bit: Layout+54-bit:Seq
  • 0 Fixed, guarante non-negative.
  • Crc, 0 padding default, use Crc8Long to gen pseudo-random numbers.
  • Layout, Provide full sequence and block sequence layouts.
    • full sequence=1bit(0) + sequence(54bit)
    • block sequence=1bit(1) + block(9bit) + sequence(45bit)
  • Seq, Dependent on the layout, the following are its constraints
    • block=block(9bit=512), distinguish different ID centers.
    • sequence=sequence(45bit|54bit), The unique incremental number within block.
    • The Range of block is [1,512], and 0 means full sequence

Since there are only 55 effective bits, the following characters are useful.

  • ID centers, up to 512. Usually one data center with one ID center.
  • Using 50k ID/s, ID can be used for 22 years, (2^45 - 1)/(365x24x3600x50,000) = 22.3
  • In the case of the max-seq layout, the above capacity increase 512 times.
  • The sequence is not time-based, so the limit and lifetime depend only on the ID consumer.
  • The sequence is process independent, so it can generate different IDs in key-value form.

The default double-buffering implementation allows Loader to guarantee a unique ASC order, as well,

  • guarantee the global block-name generates the unique ID.
  • Intra-thread id ascending order, NO guaranteed for different threads.
  • Async id fetch when the remaining reaches the threshold (80%), no lock (non-read/write lock)
  • When switching ID segments, the minimum sync block is guaranteed for the protected resources.
  • Dynamically adjust based on the speed of ID consumption , reserving 60 seconds of use.
  • When the buffered ID are exhausted, only one Loader will load while others wait or timeout.
  • Support manual preload of all available ids.
  • Support manual or timed error clearing.
  • Support manual adjustment of runtime parameters.

e2.LightIdBufferedProvider - High performance, lightweight

Lightweight locking, high-performance, double-buffered light-id provider.

Measured performance is higher than busy-wait + read/write locks and big sync block with combined locks. The bottleneck is on the IO of the loader, which needs to be optimized for maxCount depending on the consumption.

There are 3 types of threads in total, and the read threads can be upgraded to write threads and even to loader threads. At the same time, there are multiple read threads, but only unique write threads and unique load threads.

  • Read thread, normal light-id caller
  • Write thread, read thread upgraded or load thread, append segment to buffer
  • Load thread, async thread or read thread upgraded, loading segment via loader

The double buffering works as follows, which will track the id consumption and automatically control the preload amount, but not exceed maxCount.

  • If the remaining ID is less than 20%, the only async preload maximum usage in 60s or maxCount.
  • If the ID is exhausted, the read thread is upgraded to the write thread and the other read threads wait until they are woken up or timed out
  • If the read thread upgrades to the write thread, and there is a loader, this read thread will switch buffers after a spin busy waiting.

e3.LightIdUtil - lightId as long number

Util to verify and convert lightId, long and sequence. It can also change LightId's layout and seq order globally.

  • forceBlockBit - adjust the bits count of block
  • forceBlockFirst - which layout first, Block or Sequence

e4.Ulid - Simple ULID implement

Generate ULIDs quickly only withou validation and parsing. Used for sequential random numbers, global IDs, etc.

A2.F.img/ Image

  • ImageIoFix - problem-using-imageio-write-jpg-file
  • StreamJpg - BufferedImage Writing
  • Watermark - Watermark
  • ZoomRotateCrop - Zoom Rotate and Crop

A2.G.io/ IO and File system

  • CircleInputStream - Circular Read InputStream
  • CompatibleObjectStream - Use local class to deserialize if the serialVersionUID is not compatible
  • DirHasher - Local file system cannot save too many files
  • Exec - Single thread sync execution, use Apache Commons Exec for more feature
  • InputStreams - Stream operatioin when there's no commons-*
  • NonCloseStream - Ignore stream close
  • Zipper - Recursive zip/unzip

A2.H.jaxb/ xml

Note: jaxb is removed in Java 11 and need manually add the dependency.

  • StringMapXmlWriter - top-level element into a key-value map, mostly used in parameter signatures.

A2.I.lock/ Locks

  • GlobalLock - Global lock interface
  • JvmStaticGlobalLock - Jvm global lock using WeakReference

A2.J.math/ Domain's Algorithms

  • AnyIntegerUtil - int, long, Number and String
  • AverageDecimal - Average 20/6=[3.33, 3.33, 3.34, 3.33, 3.33, 3.34]
  • BalanceDecimal - Balance is Weighted Average
  • BigDecimalUtil - null friendly Decimal Utility
  • ComparableUtil - Comparator without null
  • RatioNumber - Relationship between two units and their carrying and borrowing

A2.K.netx/ Network

  • SslTrustAll - Trust all the certs to ignore crawler ssl errors
  • SslVersion - jdk-8-will-use-tls-12-as-default

A2.L.page/ Paginate

  • PageQuery - Paginated query
  • PageResult - Paginated result
  • PageUtil - Pagination with -1+1 algorithm, Not if-else

A2.M.pain/ Exception

  • BadArgsException - i18n, enum no-stackable IllegalArgumentException
  • BadStateException - i18n, enum no-stackable IllegalStateException
  • CodeException - i18n, enum no-stackable RuntimeException
  • DebugException - Only for debug or testing code
  • HttpStatusException - HttpStatus no-stackable RuntimeException
  • IllegalRequestException - IllegalRequest
  • IllegalResponseException - IllegalResponse
  • IORuntimeException - Runtime of IOException
  • MessageException - no-stackable message
  • NoStackRuntimeException - no-stack, Used for performance-first stack-useless scenarios
  • ThrowableUtil - stack and cause util
  • TimeoutRuntimeException - Runtime of TimeoutException
  • UncheckedException - wrapping a checked exception as RuntimeException
  • WaitingTimeoutException - stackless WaitingTimeoutException

A2.N.stat/ Statistics and Monitoring

  • GitStat - stats git by date author, can save it to mysql
  • JvmStat - Cpu, Mem and Thread of the JVM
  • LogStat - Collect log growth and keywords

A2.O.text/ Full/Half White Char and Formatter

  • BarString - Bar delimited string and Bar can be customized
  • BuilderHelper - null friendly fragment-less StringBuilder
  • BuilderHolder - StringBuilder to reduce mem fragmentation
  • CaseSwitcher - Camel, snake, pascal, kebab naming convertor
  • FormatUtil - support printf's %;logbak's {};message's {0}
  • FullCharUtil - Full char util
  • HalfCharUtil - Half char util
  • JsonTemplate - Make simple json with template
  • StringTemplate - No replace, good readability and performance
  • WhiteUtil - Make up for java trim, more Whitespace handling
  • Wildcard - Fast wildcard match, only ? and * are supported

A2.P.time/ Date and Time

  • DateFormatter - Thread safed more faster formatter
  • DateLocaling - LocalDateTime and TimeZone
  • DateNumber - Two-way conversion of date to number
  • DateParser - More efficient compatible date parsing
  • Sleep - Wrapper of Thread.sleep
  • SlideDate - Accounting Date Tool with OffsetClock
  • StopWatch - Stopwatch for time tracking of code
  • ThreadNow - Thread-level tuning clocks

A2.Q.tk Token and Ticket

Used for private key credentials that sometimes require central control and sometimes do not, in the scenario between the meaningless session and the popular JWT system.

Session replication and sticky are very stable and widely used in horizontal extensions such as redis and Hazelcast.

JsonWeb ecosystem is powerful in most data exchange and security requirements scenarios, but the credential is its new domain.

The applicable scenarios are RememberMe or A credential to read the result of async tasks.

In RememberMe, the biz-data can contain uid, while in sig-data the user secret and salt must be checked against the ticket. the validation must be evaluated when relogin, password change, or credential expiration.

q1.Ticket - Distrib credential with centralized authority

The features are short, expireable, kickable, sign-checkable, and have some business meaning instead of meaningless random tokens.

Where the Data suffix is the business semantics, the Part suffix is the transfer semantics, and the layout of different perspectives is,

  • Business layout: SigData + ~ + SigPart
  • SigData = PubPart + (~ + BizData)?
  • PubPart = mod + - + due + - + seq
  • BizData: business data, such as plaintext Json
  • Transfer layout: PubPart + ~ + SecPart
  • SecPart = (BizPart + ~)? + SigPart
  • BizPart: encrypted BizData
  • SigPart: signed data, sign the SigData
  • mod: predefined pattern of encryption, signature and BizPart type for deserialization. English number
  • due: expiration date, number of seconds from 1970-01-01, used to determine time expiration. Positive integer
  • seq: serial number of issue, used to determine old and new, business expiration, positive integer
  • salt - encryption or signature secret key, such as symmetric secret or asymmetric private key

The public part of business scenarios and descriptions are as follows.

  • mod The encryption and signature algorithms, data formats, user fields etc. are developed by the server or negotiated with the user

    • any = only parse text and merge fields
    • am0 = aes256(biz-data, salt) + md5(sig-data) Md5 without salt
    • am1 = aes256(biz-data, salt) + md5(sig-data + salt) Md5 with salt
    • ah1 = aes256(biz-data, salt) + HmacSha1(sig-data, salt) Hmac signature check
    • ah2 = aes256(biz-data, salt) + HmacSha256(sig-data, salt) Hmac signature check

  • due The seconds from 1970-01-01 that the credential expires in, not milliseconds.

    • Expired credentials, the client must go to the server to renew
    • The server determines its own actions, such as renew, reject, resend, etc.

  • seq Issue serial number. Increment when generating, and unchanged when renewing. if the seq is different, the credential must be reacquired.

    • When the user login, the credential is generated. When the other endpoint login again, the seq is incremented, then the old credentials can be kicked out.
    • When a credential expires without another login occurring, it can be renewed with seq for a new credential.
    • 0 means no need to verify serial number, generally used for weak privileges, such as temporary credentials

Credentials can be passed in the http header, session and url-string.

  • Without the private key, you cannot verify the signature and decryption, and you can only use the PubPart information.
  • With the private key, you can verify the credentials and obtain the business content through signature verification and decryption.
  • If you don't have a private key and you need to check the signature, you can leave it unsalted and don't use hmac signature.

q2.Default Implementation and Tool

  • AnyTicket - Can parse tickets of any mod
  • TicketHelp - basic tool for ticket parsing and verification