Typed properties in PHP

1 Comment

Typed class properties have been added in PHP 7.4 and provide a major improvement to PHP's type system. These changes are fully opt-in and non breaking to previous versions.

In this post we'll look at the feature in-depth, but first let's start by summarising the most important points:

  • They are available as of PHP 7.4, which is scheduled to be released in November of 2019
  • They are only available in classes and require an access modifier: public, protected or private; or var
  • All types are allowed, except void and callable

This is what they look like in action:

class Foo
{
    public int $a;

    public ?string $b = 'foo';

    private Foo $prop;

    protected static string $static = 'default';
}

If you're unsure about the added benefit of types, I'd recommend you reading this post first.

# Uninitialized

Before looking at the fun stuff, there's an important aspect about typed properties that's essential to talk about first.

Despite what you might think on first sight, the following code is valid:

class Foo
{
    public int $bar;
}

$foo = new Foo;

Even though the value of $bar isn't an integer after making an object of Foo, PHP will only throw an error when $bar is accessed:

var_dump($foo->bar);

Fatal error: Uncaught Error: Typed property Foo::$bar 
must not be accessed before initialization

As you can read from the error message, there's a new kind of "variable state": uninitialized.

If $bar didn't have a type, its value would simply be null. Types can be nullable though, so it's not possible to determine whether a typed nullable property was set, or simply forgotten. That's why "uninitialized" was added.

There are four important things to remember about uninitialized:

  • You cannot read from uninitialized properties, doing so will result in a fatal error.
  • Because uninitialized state is checked when accessing a property, you're able to create an object with an uninitialized property, even though its type is non-nullable.
  • You can write to an uninitialized property before reading from it.
  • Using unset on a typed property will make it uninitialized, while unsetting an untyped property will make it null.

Especially note that the following code, where an uninitialised, non-nullable property is set after constructing the object, is valid

class Foo
{
    public int $a;
}

$foo = new Foo;

$foo->a = 1;

While uninitialized state is only checked when reading the value of a property, type validation is done when writing to it. This means that you can be sure that no invalid type will ever end up as a property's value.

# Defaults and constructors

Let's take a closer look at how typed values can be initialized. In case of scalar types, it's possible to provide a default value:

class Foo
{
    public int $bar = 4;
    
    public ?string $baz = null;
    
    public array $list = [1, 2, 3];
}

Note that you can only use null as a default if the type is actually nullable. This might seem obvious, but there's some legacy behaviour with parameter defaults where the following is allowed:

function passNull(int $i = null)
{ /* … */ }

passNull(null);

Luckily this confusing behaviour is not allowed with typed properties.

Also note that it's impossible to have default values with object or class types. You should use the constructor to set their defaults.

The obvious place to initialize typed values would of course be the constructor:

class Foo
{
    private int $a;

    public function __construct(int $a)
    {
        $this->a = $a;
    }
}

But also remember what I mentioned before: it's valid to write to an uninitialized property, outside of the constructor. As long as there are nothing is reading from a property, the uninitialized check is not performed.

# Types of types

So what exactly can be typed and how? I already mentioned that typed properties will only work in classes (for now), and that they need an access modifier or the var key word in front of them.

As of available types, almost all types can be used, except void and callable.

Because void means the absence of a value, it makes sense that it cannot be used to type a value. callable however is a little more nuanced.

See, a "callable" in PHP can be written like so:

$callable = [$this, 'method'];

Say you'd have the following (broken) code:

class Foo
{
    public callable $callable;
    
    public function __construct(callable $callable)
    { /* … */ }
}

class Bar
{
    public Foo $foo;
    
    public function __construct()
    {
        $this->foo = new Foo([$this, 'method'])
    }
    
    private function method()
    { /* … */ }
}

$bar = new Bar;

($bar->foo->callable)();

In this example, $callable refers to the private Bar::method, but is called within the context of Foo. Because of this problem, it was decided not to add callable support.

It's no big deal though, because Closure is a valid type, which will remember the $this context where it was constructed.

With that out of the way, here's a list of all available types:

  • bool
  • int
  • float
  • string
  • array
  • iterable
  • object
  • ? (nullable)
  • self & parent
  • Classes & interfaces

# Coercion and strict types

PHP, being the dynamic language we love and hate, will try to coerce or convert types whenever possible. Say you pass a string where you expect an integer, PHP will try and convert that string automatically:

function coerce(int $i)
{ /* … */ }

coerce('1'); // 1

The same principles apply to typed properties. The following code is valid and will convert '1' to 1.

class Bar
{
    public int $i;
}

$bar = new Bar;

$bar->i = '1'; // 1

If you don't like this behaviour you can disabled it by declaring strict types:

declare(strict_types=1);

$bar = new Bar;

$bar->i = '1'; // 1

Fatal error: Uncaught TypeError: 
Typed property Bar::$i must be int, string used

# Type variance and inheritance

Even though PHP 7.4 introduced improved type variance, typed properties are still invariant. This means that the following is not valid:

class A {}
class B extends A {}

class Foo
{
    public A $prop;
}

class Bar extends Foo
{
    public B $prop;
}

Fatal error: Type of Bar::$prop must be A (as in class Foo)

If the above example doesn't seem significant, you should take a look at the following:

class Foo
{
    public self $prop;
}

class Bar extends Foo
{
    public self $prop;
}

PHP will replace self behind the scenes with the concrete class it refers to, before running the code. This means that the same error will be thrown in this example. The only way to handle it, is by doing the following:

class Foo
{
    public Foo $prop;
}

class Bar extends Foo
{
    public Foo $prop;
}

Speaking of inheritance, you might find it hard to come up with any good use cases to overwrite the types of inherited properties.

While I agree with that sentiment, it's worth noting that it is possible to change the type of an inherited property, but only if the access modifier also changes from private to protected or public.

The following code is valid:

class Foo
{
    private int $prop;
}

class Bar extends Foo
{
    public string $prop;
}

However, changing a type from nullable to non-nullable or reverse, is not allowed.

class Foo
{
    public int $a;
    public ?int $b;
}

class Bar extends Foo
{
    public ?int $a;
    public int $b;
}

Fatal error: Type of Bar::$a must be int (as in class Foo)

# There's more!

Like a said at the start of this post, typed properties are a major addition to PHP. There's lots more to say about them. I'd suggest you reading through the RFC to know all the neat little details.

If you're new to PHP 7.4, you probably want to read the full list of changes made and features added. To be honest, it's one of the best releases in a long time, and worth your time!

Finally, if you have any thoughts you want to share on the topic, I'd love to hear from you! You can reach me via Twitter or e-mail.

Until next time!

Read the whole story
chrisminett
2293 days ago
reply
Looking forward to it!
Milton Keynes, UK
Share this story
Delete

Git Ransom Campaign Incident Report – Atlassian Bitbucket, GitHub, GitLab

1 Share

Background and Summary of Event

Today, Atlassian Bitbucket, GitHub, and GitLab are issuing a joint blog post, in a coordinated effort to help educate and inform users of the three platforms on secure best practices relating to the recent Git ransomware incident. Though there is no evidence Atlassian Bitbucket, GitHub, or GitLab products were compromised in any way, we believe it’s important to help the software development community better understand and collectively take steps to protect against this threat.

On Thursday, May 2nd, the security teams of Atlassian Bitbucket, GitHub, and GitLab learned of a series of user account compromises across all three platforms. These account compromises resulted in a number of public and private repositories being held for ransom by an unknown actor. Each of the teams investigated and assessed that all account compromises were the result of unintentional user credential leakage by users or other third-parties, likely on systems external to Bitbucket, GitHub, or GitLab.

The security and support teams of all three companies have taken and continue to take steps to notify, protect, and help affected users recover from these events. Further, the security teams of all three companies are also collaborating closely to further investigate these events in the interest of the greater Git community. At this time, we are confident that we understand how the account compromises and subsequent ransom events were conducted. This coordinated blog post will outline the details of the ransom event, provide additional information on how our organizations protect users, and arm users with information on recovering from this event and preventing others.

Event Details

On the evening of May 2nd (UTC), all three companies began responding to reports that user repositories, both public and private, were being wiped and replaced with a single file containing the following ransom note:

> To recover your lost data and avoid leaking it: Send us 0.1 Bitcoin (BTC) to our Bitcoin address 1ES14c7qLb5CYhLMUekctxLgc1FV2Ti9DA and contact us by Email at admin@gitsbackup.com with your Git login and a Proof of Payment. If you are unsure if we have your data, contact us and we will send you a proof. Your code is downloaded and backed up on our servers. If we dont receive your payment in the next 10 Days, we will make your code public or use them otherwise.

Through immediate independent investigations, all three companies observed that user accounts were compromised using legitimate credentials including passwords, app passwords, API keys, and personal access tokens. Subsequently, the bad actor performed command line Git pushes to repositories accessible to these accounts at very high rates, indicating automated methods. These pushes overwrote the repository contents with the ransom note above and erased the commit history of the remote repository. Incident responders from each of the three companies began collaborating to protect users, share intelligence, and identify the source of the activity. All three companies notified the affected users and temporarily suspended or reset those accounts in order to prevent further malicious activity.

During the course of the investigation, we identified a third-party credential dump being hosted by the same hosting provider where the account compromise activity had originated. That credential dump comprised roughly one third of the accounts affected by the ransom campaign. All three companies acted to invalidate the credentials contained in that public dump.

Further investigation showed that continuous scanning for publicly exposed `.git/config` and other environment files has been and continues to be conducted by the same IP address that conducted the account compromises, as recently as May 10th. These files can contain sensitive credentials and personal access tokens if care is not taken to prevent their inclusion, and they should not be publicly accessible in repositories or on web servers. This problem is not a new one. More information on the `.git` directory and the `.git/config` file is available here and here. Additional IPs residing on the same hosting provider are also exhibiting similar scanning behavior. We are confident that this activity is the source of at least a portion of the compromised credentials.

Known ransom activity ceased on May 2nd. All known affected users have had credentials reset or revoked, and all known affected users have been notified by all three companies. We recommend all users take steps to protect themselves from such attacks, more information on doing so and on restoring affected repositories is available below.

How to Protect Yourself

Enable multi-factor authentication on your software development platform of choice.

Use strong and unique passwords for every service.

  • Strong and unique passwords prevent credential reuse if a third-party experiences a breach and leaks credentials.
  • Use a password manager (if approved by your organization) to make this easier!

Understand the risks associated with the use of personal access tokens.

  • Personal access tokens, used via Git or the API, circumvent multi-factor authentication.
  • Tokens may have read/write access to repositories depending on scope and should be treated like passwords.
  • If you enter your token into the clone URL when cloning or adding a remote, Git writes it to your `.git/config` file in plain text, which may carry a security risk if the `.git/config` file is publicly exposed.
  • When working with the API, use tokens as environment variables instead of hardcoding them into your programs.

Do not expose `.git` directories and `.git/config` files containing credentials or tokens in public repositories or on web servers.

How to Recover an Affected Repository

If you have a full current copy of the repository on your computer, you can force push to the current HEAD of your local copy using:

`git push origin HEAD:master –force`

Otherwise, you can still clone the repository and make use of:

Additional assistance on Git usage is available in the following resources:

Should you require additional assistance recovering your repository contents, please refer to the following:

Bitbucket:

GitHub:

GitLab:

What the Software Development Platform Community is Doing to Protect Users

All three platforms provide robust multi-factor authentication options:

Bitbucket provides the ability for admins to require 2-factor authentication and the ability to restrict access to users on certain IP addresses (IP Whitelisting) on their Premium plan.

GitHub provides token scanning to notify a variety of service providers if secrets are published to public GitHub repositories. GitHub also provides extensive guidance on preventing unauthorized account access. We encourage all users to enable two-factor authentication.

GitLab provides secrets detection in 11.9 as part of the SAST functionality. We also encourage users to enable 2FA here, and setting up ssh keys.

Thanks to the security and support teams of Atlassian Bitbucket, GitHub, and GitLab, including the following individuals for their contributions to this investigation and blog post: Mark Adams, Ethan Dodge, Sean McLucas, Elisabeth Nagy, Gary Sackett, Andrew Wurster (Atlassian Bitbucket); Matt Anderson, Howard Draper, Jay Swan, John Swanson (GitHub); Paul Harrison, Anthony Saba, Jan Urbanc, Kathy Wang (GitLab).




The post Git Ransom Campaign Incident Report – Atlassian Bitbucket, GitHub, GitLab appeared first on Bitbucket.

Read the whole story
chrisminett
2355 days ago
reply
Milton Keynes, UK
Share this story
Delete

Avengers Endgame Trailer – Developer edition [No spoilers]

1 Share
Note : Strip written without having seen the movie, so there is no spoil at all, we haven’t watch it yet.

Read the whole story
chrisminett
2373 days ago
reply
Milton Keynes, UK
Share this story
Delete

Zend Framework transitions to the Linux Foundation

1 Comment

Since its inception, Zend Technologies, and later Rogue Wave Software, has been single-handedly leading and sponsoring the Zend Framework project. Over the years, Zend Framework has seen wide adoption across the PHP ecosystem, with an emphasis on the Enterprise market. It has formed the basis of numerous business application and services including eCommerce platforms, content management, healthcare systems, entertainment platforms and portals, messaging services, APIs, and many others.

To take it to the next step of adoption and innovation, we are happy to announce that we have reached an agreement to transition Zend Framework and all its subprojects to a new open source foundation under the Linux Foundation called Laminas.

To learn more about the Laminas project and how to get involved, please visit GetLaminas.org.

The following two tabs change content below.

Matthew is an open source software architect, specializing in PHP. Currently, he is a principal engineer and the project lead for Zend Framework, a project with which he has been involved in since before the first public preview release. Matthew is a Zend Certified Engineer, and a member of the Zend Education Advisory Board, the group responsible for authoring the Zend Certification Exam. He also contributes to a number of open source projects, blogs on PHP-related topics, and presents talks and tutorials related to PHP development.

Read the whole story
chrisminett
2376 days ago
reply
Dependencies will eventually be using the Laminas namespace
Milton Keynes, UK
Share this story
Delete

PSA: Many users reporting issues signing into Google accounts through Mail on macOS 10.14.4

1 Comment

Apple yesterday released macOS 10.14.4 to the public with features such as automatic Dark Mode in Safari, Apple News+, and more. On the other hand, the update also seems to include one frustrating bug related to authenticating Gmail accounts through Apple Mail.

more…

The post PSA: Many users reporting issues signing into Google accounts through Mail on macOS 10.14.4 appeared first on 9to5Mac.

Read the whole story
chrisminett
2404 days ago
reply
Yet another Mojave issue
Milton Keynes, UK
Share this story
Delete

Un-split brain MySQL via gh-mysql-rewind

1 Comment

We are pleased to release gh-mysql-rewind, a tool that allows us to move MySQL back in time, automatically identify and rewind split brain changes, restoring a split brain server into a healthy replication chain.

I recently had the pleasure of presenting gh-mysql-rewind at FOSDEM. Video and slides are available. Consider following along with the video.

Motivation

Consider a split brain scenario: a "standard" MySQL replication topology suffered network isolation, and one of the replicas was promoted as new master. Meanwhile, the old master was still receiving writes from co-located apps.

Once the network isolation is over, we have a new master and an old master, and a split-brain situation: some writes only took place on one master; others only took place on the other. What if we wanted to converge the two? What paths do we have to, say, restore the old, demoted master, as a replica of the newly promoted master?

The old master is unlikely to agree to replicate from the new master. Changes have been made. AUTO_INCREMENT values have been taken. UNIQUE constraints will fail.

A few months ago, we at GitHub had exactly this scenario. An entire data center went network isolated. Automation failed over to a 2nd DC. Masters in the isolated DC meanwhile kept receiving writes. At the end of the failover we ended up with a split brain scenario - which we expected. However, an additional, unexpected constraint forced us to fail back to the original DC.

We had to make a choice: we've already operated for a long time in the 2nd DC and took many writes, that we were unwilling to lose. We were OK to lose (after auditing) the few seconds of writes on the isolated DC. But, how do we converge the data?

Backups are the trivial way out, but they incur long recovery time. Shipping backup data over the network for dozens of servers takes time. Restore time, catching up with changes since backup took place, warming up the servers so that they can handle production traffic, all take time.

Could we have reduces the time for recovery?

There are multiple ways to do that: local backups, local delayed replicas, snapshots... We have embarked on several. In this post I wish to outline gh-mysql-rewind, which programmatically identifies the rogue (aka "bad") transactions on the network isolated master, rewinds/reverts them, applies some bookkeeping and restores the demoted master as a healthy replica under the newly promoted master, thereby prepared to be promoted if needed.

General overview

gh-mysql-rewind is a shell script. It utilizes multiple technologies, some of which do not speak to each other, to be able to do the magic. It assumes and utilizes the following:

Some breakdown follows.

GTID

MySQL GTIDs keep track of all transactions executed on a given server. GTIDs indicate which server (UUID) originated a write, and ranges of transaction sequences. In a clean state, only one writer will generate GTIDs, and on all the replicas we would see the same GTID set, originated with the writer's UUID.

In a split brain scenario, we would see divergence. It is possible to use GTID_SUBTRACT(old_master-GTIDs, new-master-GTIDs) to identify the exact set of transactions executed on the old, demoted master, right after the failover. This is the essence of the split brain.

For example, assume that just before the network partition, GTID on the master was 00020192-1111-1111-1111-111111111111:1-5000. Assume after the network partition the new master has UUID of 00020193-2222-2222-2222-222222222222. It began to take writes, and after some time its GTID set showed 00020192-1111-1111-1111-111111111111:1-5000,00020193-2222-2222-2222-222222222222:1-200.

On the demoted master, other writes took place, leading to the GTID set 00020192-1111-1111-1111-111111111111:1-5042.

We will run...

SELECT GTID_SUBTRACT(
  '00020192-1111-1111-1111-111111111111:1-5042',
  '00020192-1111-1111-1111-111111111111:1-5000,00020193-2222-2222-2222-222222222222:1-200'
);

> '00020192-1111-1111-1111-111111111111:5001-5042'

...to identify the exact set of "bad transactions" on the demoted master.

Row Based Replication

With row based replication, and with FULL image format, each DML (INSERT, UPDATE, DELETE) writes to the binary log the complete row data before and after the operation. This means the binary log has enough information for us to revert the operation.

Flashback

Developed by Alibaba, flashback has been incorporated in MariaDB. MariaDB's mysqlbinlog utility supports a --flashback flag, which interprets the binary log in a special way. Instead of printing out the events in the binary log in order, it prints the inverted operations in reverse order.

To illustrate, let's assume this pseudo-code sequence of events in the binary log:

insert(1, 'a')
insert(2, 'b')
insert(3, 'c')
update(2, 'b')->(2, 'second')
update(3, 'c')->(3, 'third')
insert(4, 'd')
delete(1, 'a')

A --flashback of this binary log would produce:

insert(1, 'a')
delete(4, 'd')
update(3, 'third')->(3, 'c')
update(2, 'second')->(2, 'b')
delete(3, 'c')
delete(2, 'b')
delete(1, 'a')

Alas, MariaDB and flashback do not speak MySQL GTID language. GTIDs are one of the major points where MySQL and MariaDB have diverged beyond compatibility.

The output of MariaDB's mysqlbinlog --flashback has neither any mention of GTIDs, nor does the tool take notice of GTIDs in the binary logs in the first place.

gh-mysql-rewind

This is where we step in. GTIDs provide the information about what went wrong. flashback has the mechanism to generate the reverse sequence of statements. gh-mysql-rewind:

  • uses GTIDs to detect what went wrong
  • correlates those GTID entries with binary log files: identifies which binary logs actually contain those GTID events
  • invokes MariaDB's mysqlbinlog --flashback to generate the reverse of those binary logs
  • injects (dummy) GTID information into the output
  • computes ETA

This last part is worth elaborating. We have created a time machine. We have the mechanics to make it work. But as any Sci-Fi fan knows, one of the most important parts of time travel is knowing ahead where (when) you are going to land. Are you back in the Renaissance? Or are you suddenly to appear on board the French Revolution? Better dress accordingly.

In our scenario it is not enough to move MySQL back in time to some consistent state. We want to know at what time we landed, so that we can instruct the rewinded server to join the replication chain as a healthy replica. In MySQL terms, we need to make MySQL "forget" everything that ever happened after the split brain: not only in terms of data (which we already did), but in terms of GTID history.

gh-mysql-rewind will do the math to project, ahead of time, at what "time" (i.e. GTID set) our time machine arrived. It will issue a `RESET MASTER; SET GLOBAL gtid_purged='gtid-of-the-landing-time'" to make our re-winded MySQL consistent not only with some past dataset, but also with its own perception of the point in time where that dataset existed.

Limitations

Some limitations are due to MariaDB's incompatibility with MySQL, some are due to MySQL DDL nature, some due to the fact gh-mysql-rewind is a shell script.

  • Cannot rewind DDL. DDLs are silently ignored, and will impose a problem when trying to re-apply them.
  • JSON, POINT data types are not supported.
  • The logic rewinds the MySQL server farther into the past than strictly required. This simplifies the code considerably, but imposed superfluous time to rewind+reapply, i.e. time to recover.
  • Currently, this only works one server at a time. If a group of 10 servers were network isolated together, the operation would need to run on each of these 10 servers.
  • Runs locally on each server. Requires both MySQL's mysqlbinlog as well as MariaDB's mysqlbinlog.

Testing

There's lot of moving parts to this mechanism. A mixture of technologies that don't normally speak to each other, injection of data, prediction of ETA... How reliable is all this?

We run continuous gh-mysql-rewind testing in production to consistently prove that it works as expected. Our testing uses a non-production, dedicated, functional replica. It contaminates the data on the replica. It lets gh-mysql-rewind automatically move it back in time, it joins the replica back into the healthy chain.

That's not enough. We actually create a scenario where we can predict, ahead of testing, what the time-of-arrival will be. We checksum the data on that replica at that time. After contaminating and effectively breaking replication, we expect gh-mysql-rewind to revert the changes back to our predicted point in time. We checksum the data again. We expect 100% match.

See the video or slides for more detail on our testing setup.

Status

At this time the tool in one of several solutions we hope to never need to employ. It is stable and tested. We are looking forward to a promising MySQL development that will provide GTID-revert capabilities using standard commands, such as SELECT undo_transaction('00020192-1111-1111-1111-111111111111:5042').

We have released gh-mysql-rewind as open source, under the MIT license. The public release is a stripped down version of our own script, which has some GitHub-specific integration. We have general ideas in incorporating this functionality into higher level tools.

gh-mysql-rewind is developed by the database-infrastructure team at GitHub.

Read the whole story
chrisminett
2407 days ago
reply
Very interesting way to resolve split brain. Unfortunately we can't use due to requirement for MariaDB.
Milton Keynes, UK
Share this story
Delete
Next Page of Stories