Axion

Haskell

Package Highlight: newtype-generics

A package highlight on newtype-generics, safe coercions, and a smaller type-family approach to recovering underlying newtype representations.

In my Package Highlight series, I am writing about the packages I found interesting.

newtype-generics is a package for determining the underlying type of a newtype.

In Haskell, there is no good built-in method for calculating the underlying type of a newtype. This proves necessary sometimes.

This package defines the Newtype type class (GitHub source), which relates a newtype to its underlying representation. It doesn’t use any hacks to find the base type; rather, programmers should volunteer an instance of this type class where the base type is associated. (See Associated Types, “Type Families in Haskell: The Definitive Guide”, Serokell).

-- https://github.com/sjakobi/newtype-generics/blob/a350472e09f4cb344e4fee3fcfc95b7668c7420d/Control/Newtype/Generics.hs#L100C1-L118C26

-- Original Newtype class, extended with generic defaults (trivial) and deprived
-- of the second type argument (less trivial, as it involves a type family with
-- a default, plus an equality constraint for the related type family in
-- GNewtype). We do get rid of MultiParamTypeClasses and FunctionalDependencies,
-- though.

-- | As long as the type @n@ is an instance of Generic, you can create an instance
-- with just @instance Newtype n@
class Newtype n where
  type O n :: Type
  type O n = GO (Rep n)

  pack   :: O n -> n
  default pack :: (Generic n, GNewtype (Rep n), O n ~ GO (Rep n)) => O n -> n
  pack = to . gpack

  unpack :: n -> O n
  default unpack :: (Generic n, GNewtype (Rep n), O n ~ GO (Rep n)) => n -> O n
  unpack = gunpack . from

Is that useful? Sure, in a niche situation: whenever you need to “downcast” a newtype, whereas the context only fixes the type up to an ad hoc constraint (e.g., type class). If that sounds like alien talk, that’s understandable. But it comes up so often enough if I showed you an example, you’d probably recognize it.

Consider serialization. Often, there are multiple ways to encode the same data type. As an example, some Int32 fields may be encoded using a LEB128 encoding, while others may be transmitted in a big-endian format. While it’s convenient to use a generically derived serializer, but what should we do when we must use variant encoding?

-- Example: Two fields of the same type, encoded differently.
data Packet123 = Packet123
  { field0 :: Int32 -- big-endian, 4 bytes
  , field1 :: Int32 -- LEB128
  } deriving (Generic)

instance Binary Packet123 -- Oops! Both are encoded as big-endian numbers!

Let’s annotate our fields with the choice of encoding, since it’s undesirable to give up automatic derivation. The canonical approach is wrapping them in newtypes. (Read “Newtype”, the HaskellWiki. A newtype is exactly represented by the base type, so “casting” is always a no-op. For information about coercions, read “Safe Zero-cost Coercions for Haskell”, Breitner et al.)

-- (Data.Binary is from the 'binary' package, a serialization library)
import Data.Binary

-- Package base
import Data.Bits
import Data.Functor.Identity

-- binary's pass-through stock instance for Identity
-- instance Binary a => Binary (Identity a)
-- Use the LEB128 encoding for integral types.
newtype LEB128 a = MkLEB128 a deriving (Generic)

-- Define the encoding.
instance (FiniteBits a, Integral a) => Binary (LEB128 a) where
  put = _
  get = _

-- Annotate the encoding.
data Packet123 = Packet123
  { field0 :: Identity Int32 -- Big-endian encoding (stock 'binary' instance)
  , field1 :: LEB128 Int32   -- Our alternative encoding of the same type
  }

Now the users don’t get clean Int32s, but newtypes over Int32s. Safe coercions can assist uniformly unpacking the newtypes, but we hit a problem when we apply polymorphism to the base type.

-- Ad hoc polymorphism using a type class. As we'll see,
-- it doesn't give enough information for 'coerce' to determine
-- what to coerce the base type to.
action :: (Integral i) => i -> i -> MyMonad Bool

-- Illustration of ambiguity when using 'coerce' and how
-- 'unpack' is different.
usePacket123 Packet123 { field0, field1 } = do
  let realField0 = unpack field0 -- like coerce @(Identity Int32) @Int32
      realField1 = unpack field1 -- like coerce @(LEB128 Int32) @Int32
  action realField0 realField1

coerce commutes any pair of types with the same representation once the types are given. But it doesn’t help us find and downcast to the base type. Using coerce here fails because the target type is ambiguous. coerce needs both types to be known, but the context only constrains the result to satisfy Integral. coerce is the wrong tool for the job.

The right thing to do is use unpack: it functionally determines the underlying type and downcasts the newtype. This is such a common problem in some modules, I’ve found this fix highly useful.

So, do I recommend this library? Not really - here’s why.

The library was useful, but it’s less relevant today, and maintenance has slowed down. The latest version (0.6.2 as of writing in July, 2025) of newtype-generics can’t build with the latest GHC (version 9.12.*; base 4.21).

While this package was once useful, GHC’s safe coercion feature replaced most of this, except the Newtype type class. And, the type families feature may have been a simpler replacement for the type class all along.

Here’s the timeline:

  • In 2007-2008, GHC introduced type families and type equality (~). Note: it was already possible, in theory, to use an open type family instead of a type class (Newtype).
  • In around 2011, Darius Jahandarie published the newtype package, a predecessor to this package. He introduced the Newtype type class, and combinators to cast functions to work over or under newtypes. (For example, if Age was an Int newtype, it could cast f :: Int -> Int as Age -> Age and vice versa, among other combinators.)
  • In 2014, GHC added safe coercions, rendering the conversion combinators obsolete.
  • In the same year, sjakobi (Simon Jakobi) forked newtype and republished it as newtype-generics. sjakobi fixed typing issues in newtype by using an associated type (instead of functional dependencies). He also added generic derivation, hence the name of the package.

Instead of this package and the type class, we can use a simple type family-based solution like this.

-- |
-- Module: Control.Newtype.Class
-- Description: Functionally determine the original type of newtypes (when registered).
--
-- This entire self-contained module registers the original type of newtypes using an open type family.
--
-- The `newtype` and `newtype-generics` packages' functions aren't included because they're easily made with 'Data.Coerce.coerce'. See:
--
-- @
-- -- https://github.com/love-haskell/coercible-utils/blob/0c634f98af9e492960521e481dcb781759198f46/src/CoercibleUtils/Newtype.hs
-- op _ = 'Data.Coerce.coerce'
-- ala' _ = 'Data.Coerce.coerce'
-- @
module Control.Newtype.Class
  ( NewtypeOf,
    Newtyped,
    Newtype,
    pack,
    unpack,
  )
where

import Data.Coerce (Coercible, coerce)
import Data.Functor.Const (Const)
import Data.Functor.Identity (Identity)

-- | An open type synonym family to associate a newtype with its underlying representation.
type family NewtypeOf a

-- | This convenience constraint can act like the original `Newtype`
-- type class, witnessing the existence of a type family instance.
type Newtype n = (NewtypeOf n ~ NewtypeOf n)

-- Internal convenience constraint
type Newtyped n o = (Coercible n o, NewtypeOf n ~ o)

-- | Unpack into the underlying (original) type.
unpack :: (Newtyped n o) => n -> o
unpack = coerce

-- | Pack into the newtype
pack :: (Newtyped n o) => o -> n
pack = coerce

-- That's it! For completeness, we'll register several base types.

type instance NewtypeOf (Identity a) = a

type instance NewtypeOf (Const a x) = a

-- add more ... many newtypes exist in Data.Semigroup and Data.Monoid.
-- Now it's time to use the things we've defined.
import Control.Newtype.Class

-- I need the original type to be functionally determined from the newtype.
consume :: (Show s) => s -> Int
consume = length . show

-- Error (type ambiguity; Show s is not enough to determine
-- what to coerce 'Identity String' to.)
hello1 :: Identity String -> Int
hello1 = consume . coerce

-- No ambiguity; the `NewtypeOf` constraint
-- specifies that `NewtypeOf (Identity String) = String`.
hello2 :: Identity String -> Int
hello2 = consume . unpack
Type error (details) 
• Ambiguous type variable ‘b0’ arising from a use of ‘consume’
  prevents the constraint ‘(Show b0)’ from being solved.
  Probable fix: use a type annotation to specify what ‘b0’ should be.
  Potentially matching instances:
    instance Show a => Show (Identity a)
      -- Defined in ‘ghc-internal-9.1002.0:GHC.Internal.Data.Functor.Identity’
    instance Show Ordering
      -- Defined in ‘ghc-internal-9.1002.0:GHC.Internal.Show’
    ...plus 26 others
    ...plus 89 instances involving out-of-scope types
    (use -fprint-potential-instances to see them all)
• In the first argument of ‘(.)’, namely ‘consume’
  In the expression: consume . coerce
  In an equation for ‘hello1’: hello1 = consume . coerce

• Couldn't match representation of type ‘b0’ with that of ‘String’
    arising from a use of ‘coerce’
• In the second argument of ‘(.)’, namely ‘coerce’
  In the expression: consume . coerce
  In an equation for ‘hello1’: hello1 = consume . coerce

Let me note one important change, though. The old Newtype type class forces open the base types - the associated type (O _) is public. It also exposes pack and unpack class methods, hindering encapsulation. I think it’s an unfortunate side effect of using a type class. (But, you can require an extra Coercible constraint to the pack and unpack to improve the situation. The compiler exposes Coercible only when a newtype constructor is visible.)

That’s not a problem with my type-family approach as it requires a visible Coercible instance. By design, if you hide the constructor, you’ll limit access to pack and unpack. If that’s a problem (though, I can’t see why that’d be a problem), using a type class with an associated type restores the original behavior.

Using a type family that drops information might be useful if GHC’s generic mechanism provided that metainformation. So, if I could tell the two data declarations apart using Generic, it should be possible to implement a true annotation system (like in Go, C#, and so on.).

type family Ann (d :: k) t where
  Ann _ t = t

-- Can we tell the difference using 'Generic'? If so,
-- can we extract the information?
-- (I haven't experimented with it yet, so I don't know,
-- but I'd like to know.)

data Data1 = Data1
  { d1f1 :: Ann "Hi" Int
  , d1f2 :: Ann Identity Float
  } deriving (Generic)

data Data2 = Data2
  { d2f1 :: Int
  , d2f2 :: Float
  } deriving (Generic)

In conclusion, should you use this package? No, you don’t; Haskell itself gained language features that made it obsolete. Regardless, I think it’s an artifact from an interesting moment in Haskell’s history.